Tutorials - Big bits of code to help you do more
Paginating a filtered DataObjectSet
Tweet25 October 2010 | | | Supports v2.4
If you have ever tried to paginate a DataObjectSet that has been customised since fetching it from the database you will have descovered that it simply doesn't work. This short tutorial provides some great code to resolve this problem and paginate to your hearts content!
The problem
In its standard implementation DataObjectSet does not allow pagination of a set that has been filtered after the set has been fetched from the database.So if you have a function called getNewsArticles() in your model that looks like this“
function getNewsArticles() {
$newsArticles = DataObject::get("NewsArticle");
$doSet = new DataObjectSet();
foreach ($newsArticles as $newsArticle) {
if ($newsArticle->canView()) {
$doSet->push($newsArticle);
}
}
return $doSet;
}
Then you can't paginate it using DataObjectSets built in pagination controls. The built in controls make the assumption that the set that's being paginated is the set that has been fetched from the database and that the SQL LIMIT clause is the basis for the page size. When you filter a set after fetching it from the database then using the LIMIT clause won't work because we don't know how many records we'll end up with after filtering.
The solution
In order to make filtered pagination work we will decorate DataObjectSet with our own pagination method. When you decorate a class all instantiated objects of that class will have this method available for use. The decorated method will slice out the records we need for the current page. The decorator looks like this
class DataObjectSetExtension extends Extension {
public function Pagination() {
$pageLimits = $this->owner->getPageLimits();
$items = $this->owner->toArray();
$items = array_slice($items, $pageLimits["pageStart"], $pageLimits["pageLength"]);
return new DataObjectSet($items);
}
}
What the code does is fetch the records as an array from the owner class (in this case DataObjectSet) and then slice out the records it needs based on the pageLimits set on the DataObjectSet.
In order to make the method available on all our DataObjectSets the following line needs to be put in _config.php
Object::add_extension('DataObjectSet', 'DataObjectSetExtension');
After flushing your site the method will now be available on all DataObjectSets. In order to get the getNewsArticles() function to work with our pagination it nees to be modified. The modified function looks like this
function getNewsArticles() {
if(!isset($_GET['start']) || !is_numeric($_GET['start']) || (int)$_GET['start'] < 1) $_GET['start'] = 0;
$SQL_start = (int)$_GET['start'];
$newsArticles = DataObject::get("NewsArticle");
$doSet = new DataObjectSet();
foreach ($newsArticles as $newsArticle) {
if ($newsArticle->canView()) {
$doSet->push($newsArticle);
}
}
$doSet->setPageLimits($SQL_start, 30, $doSet->Count());
return $doSet;
}
As you can see, some things have changed. First of all, we're checking for a GET variable called start. This is the standard pagination variable for DataObjectSet but you can change it if you want with DataObjectSet->setPaginationGetVar($var).
The fetching and filtering is done exactly as before but before returning the set we need to explicitly set the page limits with the setPageLimits() function. If setPageLimits() is not set then the Pagination method won't know what values to use. In this case I have set the start value to the GET variable, page size to 30 and the total size to the number of records in the set after the filtering has been done.
When displaying the news articles in the template we loop over the Pagination method of the DataObjectSet instead of just looping over the getNewsArticles() function like we normally do. Note that you only use the Pagination method for printing the actual results, not for creating the pagination controls. The code below is mostly taken from the SilverStripe pagination tutorial that can be found here.
<% control NewsArticles.Pagination %>
<!--
Put whatever you want to display here
-->
<% end_control %>
<% if NewsArticles.MoreThanOnePage %>
<p>
<% if NewsArticles.PrevLink %>
<a href="$NewsArticles.PrevLink"><&;lt; Prev</a> |
<% end_if %>
<% control NewsArticles.Pages %>
<% if CurrentBool %>
<strong>$PageNum</strong>
<% else %>
<a href="$Link" title="Go to page $PageNum">$PageNum</a>
<% end_if %>
<% end_control %>
<% if NewsArticles.NextLink %>
| <a href="$NewsArticles.NextLink">Next >></a>
<% end_if %>
</p>
<% end_if %>
As you can see in the snippet above we still use the regular set for the pagination controls but we use the Pagination method for actually displaying the records.
So to recap
- Put the DataObjectSetExtension.php in your code folder
- Add the line Object::add_extension('DataObjectSet', 'DataObjectSetExtension') to _config.php and flush your site
- Fetch the data, filter it and set the page limits
- Use the Pagination method to show your stuff
That's all, happy coding!
13 Comments
RSS feed for comments on this page RSS feed for all comments
joel
25/10/2010 9:56am (3 years ago)
Hi Marcus, this is really awesome and useful! Thanks a hundred times.
Marcus Dalgren
25/10/2010 10:00am (3 years ago)
Thank you Joel! Really nice to hear that someone else needs this as much as I did when I had to figure this out. Let me know if you need any help or have any questions.
Daniel Hensby
26/10/2010 9:01am (3 years ago)
Hi Marcus,
Nice tut and really useful.
However, I can't help but notice that you will iterate over *every* DO returned despite the fact you might only really need 10 (to show). With a small set, this wouldn't be a massive issue; but if you wanted to paginate a set of thousands, this could be quite slow. Have you thought of a way to get around this problem?
Marcus Dalgren
26/10/2010 9:10am (3 years ago)
Hi Dan,
Unfortunately no, I don't think there is an easy way around this unless you cache the initial query and filtering in some way. The big issue is that you're filtering after the results have been fetched, presumably because you can't filter the result directly in the database query.
If you can filter in the database query that's of course alot better but there needs to be some kind of solution for when you can't too. Is there a simple way to cache a DataObjectSet in SilverStripe? If there is then let me know and we can add that to the tutorial. It would be really nice to cache the filtered result after the first query and then just use that on subsequent calls.
Has anyone tried caching DataObjectSets and knows how to go about this?
Martijn van Nieuwenhoven
27/10/2010 1:58am (3 years ago)
You can take a look at Aram's custom search mod hidden in the forums.
http://www.silverstripe.org/all-other-modules/show/6641?start=24#post290089
Specially the part from r303 at $records = DB::query($fullQuery);
After the query the result will be pushed in a DO set as well, but its much smaller, since its not getting the full DataObjects.
That still does not prevent you checking canView on each DO though...
Daniel Hensby
27/10/2010 1:06pm (3 years ago)
@Marcus
Yes, there is a very simple way to cache an output of a function. I use two methods, one is this in the init() of a controller:
$this->FuncName = $this->cacheToFile('FuncName',$lifetime,$ID,(array)$arguments);
The other is this:
function() FuncName() {
if (!isset($this->FuncName) {
$this->FuncName = some complicated fetching etc;
}
return $this->FuncName;
I might write a lil tut on caching in the contoller
Marcus Dalgren
27/10/2010 2:35pm (3 years ago)
@Daniel
That sounds pretty awesome. Would you just cache the results from the query and filtering and live with the cost of slicing a really big array or would you try to find a way to cache each page separately? Also how big would you let the result set get before you'd seriously start considering going the cache route?
Just caching the results after the filtering seems to be pretty trivial with cacheToFile() if I'm reading the code correctly. Thank you for the awesome tip, I will be trying it out soon.
Manuel
05/11/2010 8:16pm (3 years ago)
Hi, my solution for pagination of custom DataObjectSets looks like this:
public function getChildPages() {
$kids = DataObject::get('ViewableObjectsHolder', "ParentID = '{$this->ID}'", 'Sort');
foreach ($kids as $kid) {
$kid->controller = ModelAsController::controller_for($kid);
}
return $kids;
}
public function getChildObjects($items_per_page = 10) {
$objects = new DataObjectSet();
$kids = $this->getChildPages();
foreach ($kids as $kid) {
$objects->merge($kid->controller->getObjects('', false));
}
if (empty($items_per_page) || $items_per_page < 1) $items_per_page = 10;
if(!isset($_GET['start']) || !is_numeric($_GET['start']) || (int)$_GET['start'] < 1) $_GET['start'] = 0;
$o = $objects->getRange((int)$_GET['start'], (int)$items_per_page);
$o->setPageLimits((int)$_GET['start'], (int)$items_per_page, $objects->Count());
return $o;
}
The important lines are the last ones of the 'getChildObjects' method.
Hope it helps...
Marcus Dalgren
05/11/2010 9:31pm (3 years ago)
@Manuel Man I'd actually missed the getRange() method! I could have sworn I searched for an equivalent to array_slice() but couldn't find it but it's right there.
Manuels way of doing it actually makes more sense, it becomes easier in the template since you'll always be using the same object for all of it and you skip the __call() so it should be faster as well.
Rick
05/12/2010 4:55am (3 years ago)
Sorry - just to clarify I'm doing things right:
- Manuel's code makes all your code obsolete
- you place it in a PageController (or equivalent)
- you can access different pages through the url with ?start= (is there a cleaner ss way of doing this?)
Am I right?
Cheers & thanks
Marcus Dalgren
05/12/2010 11:28am (3 years ago)
Yes pretty much.
You have to use Manuels code in every getter where you want to paginate a DataObjectSet that needs to be filtered after the db query.
You could also write a helper function with the stuff that Manuel suggests and use that every time you need to do this.
I'd do a method called paginate() that accepts the relevant DOSet and returns the paginated version. That method would keep track of the GET variables etc so you wouldn't have to clutter the getter with that stuff.
I'll ask Aram if it's ok and if it is, I'll change this tutorial to how I'd actually do it today.
Rick
05/12/2010 8:24pm (3 years ago)
Thanks Marcus!
That's very helpful. Nice tutorial writing too - it will be great if you get the go-ahead to revise it.
aleks
18/03/2011 1:57pm (2 years ago)
Thanks for this! Led me the way to the light!
By the way: the link to the SilverStripe pagination tutorial/recipe in the article is broken, it currently resides here: http://doc.silverstripe.org/old/private:recipes:pagination
Post a comment ...
You cannot post comments until you have logged in. Login Here.