SSbits - Home page
Site by Carbon Crayon
Submit a Post >

Tutorials - Big bits of code to help you do more

Paginating a filtered DataObjectSet

 Source files (336 bytes)

If you have ever tried to paginate a DataObjectSet that has been customised since fetching it from the database you will have descovered that it simply doesn't work. This short tutorial provides some great code to resolve this problem and paginate to your hearts content!

The problem

In its standard implementation DataObjectSet does not allow pagination of a set that has been filtered after the set has been fetched from the database.So if you have a function called getNewsArticles() in your model that looks like thisĀ“

function getNewsArticles() {
$newsArticles = DataObject::get("NewsArticle");
$doSet = new DataObjectSet();
foreach ($newsArticles as $newsArticle) {
if ($newsArticle->canView()) {
$doSet->push($newsArticle);
}
}
return $doSet;
}

Then you can't paginate it using DataObjectSets built in pagination controls. The built in controls make the assumption that the set that's being paginated is the set that has been fetched from the database and that the SQL LIMIT clause is the basis for the page size. When you filter a set after fetching it from the database then using the LIMIT clause won't work because we don't know how many records we'll end up with after filtering.

The solution

In order to make filtered pagination work we will decorate DataObjectSet with our own pagination method. When you decorate a class all instantiated objects of that class will have this method available for use. The decorated method will slice out the records we need for the current page. The decorator looks like this

class DataObjectSetExtension extends Extension {
public function Pagination() {
$pageLimits = $this->owner->getPageLimits();
$items = $this->owner->toArray();
$items = array_slice($items, $pageLimits["pageStart"], $pageLimits["pageLength"]);
return new DataObjectSet($items);
}
}

What the code does is fetch the records as an array from the owner class (in this case DataObjectSet) and then slice out the records it needs based on the pageLimits set on the DataObjectSet.

In order to make the method available on all our DataObjectSets the following line needs to be put in _config.php

Object::add_extension('DataObjectSet', 'DataObjectSetExtension');

After flushing your site the method will now be available on all DataObjectSets. In order to get the getNewsArticles() function to work with our pagination it nees to be modified. The modified function looks like this

function getNewsArticles() {
if(!isset($_GET['start']) || !is_numeric($_GET['start']) || (int)$_GET['start'] < 1) $_GET['start'] = 0;
$SQL_start = (int)$_GET['start'];
$newsArticles = DataObject::get("NewsArticle");
$doSet = new DataObjectSet();
foreach ($newsArticles as $newsArticle) {
if ($newsArticle->canView()) {
$doSet->push($newsArticle);
}
}
$doSet->setPageLimits($SQL_start, 30, $doSet->Count());
return $doSet;
}

As you can see, some things have changed. First of all, we're checking for a GET variable called start. This is the standard pagination variable for DataObjectSet but you can change it if you want with DataObjectSet->setPaginationGetVar($var).

The fetching and filtering is done exactly as before but before returning the set we need to explicitly set the page limits with the setPageLimits() function. If setPageLimits() is not set then the Pagination method won't know what values to use. In this case I have set the start value to the GET variable, page size to 30 and the total size to the number of records in the set after the filtering has been done.

When displaying the news articles in the template we loop over the Pagination method of the DataObjectSet instead of just looping over the getNewsArticles() function like we normally do. Note that you only use the Pagination method for printing the actual results, not for creating the pagination controls. The code below is mostly taken from the SilverStripe pagination tutorial that can be found here.

<% control NewsArticles.Pagination %>
<!--
Put whatever you want to display here
-->
<% end_control %>
<% if NewsArticles.MoreThanOnePage %>
<p>

<% if NewsArticles.PrevLink %>
<a href="$NewsArticles.PrevLink">&lt;&;lt; Prev</a> |
<% end_if %>

<% control NewsArticles.Pages %>
<% if CurrentBool %>
<strong>$PageNum</strong>
<% else %>
<a href="$Link" title="Go to page $PageNum">$PageNum</a>
<% end_if %>
<% end_control %>

<% if NewsArticles.NextLink %>
| <a href="$NewsArticles.NextLink">Next &gt;&gt;</a>
<% end_if %>

</p>
<% end_if %>

As you can see in the snippet above we still use the regular set for the pagination controls but we use the Pagination method for actually displaying the records.

So to recap

  1. Put the DataObjectSetExtension.php in your code folder
  2. Add the line Object::add_extension('DataObjectSet', 'DataObjectSetExtension') to _config.php and flush your site
  3. Fetch the data, filter it and set the page limits
  4. Use the Pagination method to show your stuff

That's all, happy coding!

Marcus Dalgren avatar

Marcus Dalgren

Marcus is a web developer currently working at a small startup company called Overlay in Gothenburg, Sweden. He also does freelance work for small to medium sized businesses in the Gothenburg area.

  • joel
    25/10/2010 9:56am (4 years ago)

    Hi Marcus, this is really awesome and useful! Thanks a hundred times.

  • Marcus Dalgren
    25/10/2010 10:00am (4 years ago)

    Thank you Joel! Really nice to hear that someone else needs this as much as I did when I had to figure this out. Let me know if you need any help or have any questions.

  • Daniel Hensby
    26/10/2010 9:01am (4 years ago)

    Hi Marcus,

    Nice tut and really useful.

    However, I can't help but notice that you will iterate over *every* DO returned despite the fact you might only really need 10 (to show). With a small set, this wouldn't be a massive issue; but if you wanted to paginate a set of thousands, this could be quite slow. Have you thought of a way to get around this problem?

  • Marcus Dalgren
    26/10/2010 9:10am (4 years ago)

    Hi Dan,

    Unfortunately no, I don't think there is an easy way around this unless you cache the initial query and filtering in some way. The big issue is that you're filtering after the results have been fetched, presumably because you can't filter the result directly in the database query.

    If you can filter in the database query that's of course alot better but there needs to be some kind of solution for when you can't too. Is there a simple way to cache a DataObjectSet in SilverStripe? If there is then let me know and we can add that to the tutorial. It would be really nice to cache the filtered result after the first query and then just use that on subsequent calls.

    Has anyone tried caching DataObjectSets and knows how to go about this?

  • Martijn van Nieuwenhoven
    27/10/2010 1:58am (4 years ago)

    You can take a look at Aram's custom search mod hidden in the forums.

    http://www.silverstripe.org/all-other-modules/show/6641?start=24#post290089

    Specially the part from r303 at $records = DB::query($fullQuery);

    After the query the result will be pushed in a DO set as well, but its much smaller, since its not getting the full DataObjects.

    That still does not prevent you checking canView on each DO though...

  • Daniel Hensby
    27/10/2010 1:06pm (4 years ago)

    @Marcus

    Yes, there is a very simple way to cache an output of a function. I use two methods, one is this in the init() of a controller:

    $this->FuncName = $this->cacheToFile('FuncName',$lifetime,$ID,(array)$arguments);

    The other is this:
    function() FuncName() {
    if (!isset($this->FuncName) {
    $this->FuncName = some complicated fetching etc;
    }
    return $this->FuncName;

    I might write a lil tut on caching in the contoller

  • Marcus Dalgren
    27/10/2010 2:35pm (4 years ago)

    @Daniel

    That sounds pretty awesome. Would you just cache the results from the query and filtering and live with the cost of slicing a really big array or would you try to find a way to cache each page separately? Also how big would you let the result set get before you'd seriously start considering going the cache route?

    Just caching the results after the filtering seems to be pretty trivial with cacheToFile() if I'm reading the code correctly. Thank you for the awesome tip, I will be trying it out soon.

  • Manuel
    05/11/2010 8:16pm (4 years ago)

    Hi, my solution for pagination of custom DataObjectSets looks like this:

    public function getChildPages() {
    $kids = DataObject::get('ViewableObjectsHolder', "ParentID = '{$this->ID}'", 'Sort');
    foreach ($kids as $kid) {
    $kid->controller = ModelAsController::controller_for($kid);
    }
    return $kids;
    }

    public function getChildObjects($items_per_page = 10) {
    $objects = new DataObjectSet();
    $kids = $this->getChildPages();
    foreach ($kids as $kid) {
    $objects->merge($kid->controller->getObjects('', false));
    }
    if (empty($items_per_page) || $items_per_page < 1) $items_per_page = 10;
    if(!isset($_GET['start']) || !is_numeric($_GET['start']) || (int)$_GET['start'] < 1) $_GET['start'] = 0;

    $o = $objects->getRange((int)$_GET['start'], (int)$items_per_page);
    $o->setPageLimits((int)$_GET['start'], (int)$items_per_page, $objects->Count());
    return $o;
    }

    The important lines are the last ones of the 'getChildObjects' method.

    Hope it helps...

  • Marcus Dalgren
    05/11/2010 9:31pm (4 years ago)

    @Manuel Man I'd actually missed the getRange() method! I could have sworn I searched for an equivalent to array_slice() but couldn't find it but it's right there.

    Manuels way of doing it actually makes more sense, it becomes easier in the template since you'll always be using the same object for all of it and you skip the __call() so it should be faster as well.

  • Rick
    05/12/2010 4:55am (4 years ago)

    Sorry - just to clarify I'm doing things right:

    - Manuel's code makes all your code obsolete
    - you place it in a PageController (or equivalent)
    - you can access different pages through the url with ?start= (is there a cleaner ss way of doing this?)

    Am I right?

    Cheers & thanks

  • Marcus Dalgren
    05/12/2010 11:28am (4 years ago)

    Yes pretty much.

    You have to use Manuels code in every getter where you want to paginate a DataObjectSet that needs to be filtered after the db query.

    You could also write a helper function with the stuff that Manuel suggests and use that every time you need to do this.

    I'd do a method called paginate() that accepts the relevant DOSet and returns the paginated version. That method would keep track of the GET variables etc so you wouldn't have to clutter the getter with that stuff.

    I'll ask Aram if it's ok and if it is, I'll change this tutorial to how I'd actually do it today.

  • Rick
    05/12/2010 8:24pm (4 years ago)

    Thanks Marcus!
    That's very helpful. Nice tutorial writing too - it will be great if you get the go-ahead to revise it.

  • aleks
    18/03/2011 1:57pm (4 years ago)

    Thanks for this! Led me the way to the light!

    By the way: the link to the SilverStripe pagination tutorial/recipe in the article is broken, it currently resides here: http://doc.silverstripe.org/old/private:recipes:pagination

Post a comment ...

You cannot post comments until you have logged in. Login Here.

Advertisement

Site of the Month

Find SSbits on

Top Contributers

Rank Avatar Name
1 article image Aram Balakjian
2 article image Daniel Hensby
3 article image Marcus Dalgren
4 article image Hamish Campbell
5 article image Ty Barho
6 article image Martijn van Nieuwenhoven
7 article image Darren-Lee
8 article image Roman Schmid
9 article image Matt Clegg
10 article image dalesaurus

View full leaderboard


Advertisement