Domain objects as JSON - Why?

If you have developed a TYPO3 extension based on extbase the time may come where you want an accessible JSON representation of all or some parts of your domain. You might encounter one of the two usecases I am going to introduce shortly or you might have another valid reason (which is perfectly possbile) or maybe you dont but want to know nevertheless (which is perfectly fine). So the usecases I had so far:

  • AJAX call: I need some data to be loaded asynchronously for the little JavaScript I wrote. E.g. there is a list of objects and the user has some filter available to reduce or change the displayed results. And I (being a smart developer) does not want a reload of the page taking place each time a filter is applied or changed. Therefore I fetch the new results via AJAX - and who does not love to get JSON back from an AJAX roundtrip?
  • API: I take one step further and want to provide a (RESTful) API that can be consumed by my JavaScript application (that may have evolved to angularJS ) and my android app and the REST (haha) of the world. In this case JSON might be a good idea as well.

 So how do we do this?

Top

Domain objects as JSON - How?

The following code examples are taken from an example extension I put on github.

You very likely are familiar with the standard way of doing an extbase Extension. Fetching your objects in your controller by kindly asking the repository and then give the received data alongside the best wishes to your fluid template that will produce the HTML output suitable for the current request.

Lets have a look at a simple domain consisting of a Tag object that only has a title:

use TYPO3\CMS\Extbase\DomainObject\AbstractEntity;

class Tag extends AbstractEntity
{
    /**
     * @var string
     */
    protected $title = '';

    // Getter and Setter
}

We have a simple plugin configured in our ext_localconf.php that displays our tags:

\TYPO3\CMS\Extbase\Utility\ExtensionUtility::configurePlugin(
    'DanielGoerz.' . $_EXTKEY,
    'json_tag',
    [
        'Tag' => 'list, show',
    ],
    []
);

Our controller just has a list and a show action:

use DanielGoerz\JsonExample\Domain\Model\Tag;
use TYPO3\CMS\Extbase\Mvc\Controller\ActionController;

class TagController extends ActionController
{
    /**
     * @var \DanielGoerz\JsonExample\Domain\Repository\TagRepository
     * @inject
     */
    protected $tagRepository;

    /**
     * @return void
     */
    public function listAction()
    {
        $this->view->assign('tags', $this->tagRepository->findAll());
    }

    /**
     * @param Tag $tag
     * @return void
     */
    public function showAction(Tag $tag)
    {
        $this->view->assign('tag', $tag);
    }
}

So far no news. This is a very basic extbase plugin.

Top

The JSON view

So lets take this example and assume the current request asks for the JSON representation of whatever came out of the repository. One way of achieving this is to make use of the class \TYPO3\CMS\Extbase\Mvc\View\JsonView (or in short: the JSON view) that extbase provides for such a task. The JSON view has been around since 6.2 where it was backported from flow by Jan Kiesewetter as a last minute feature before the LTS release.

Before we investigate what it does, lets see how we use it. In our controller we can change the member variable $defaultViewObjectName that is set to \TYPO3\CMS\Fluid\View\TemplateView::class by default in the \TYPO3\CMS\Extbase\Mvc\Controller\ActionController. We do this by simply overwriting the variable in our own controller:

class TagController extends ActionController
{
    /**
     * @var \TYPO3\CMS\Extbase\Mvc\View\JsonView
     */
    protected $view;

    /**
     * @var string
     */
    protected $defaultViewObjectName = \TYPO3\CMS\Extbase\Mvc\View\JsonView::class;

    // Rest of the code stays the same
}

In this example we have also overwritten the variable $view but we changed only the class type in the @var annotation so our IDE will know all methods our view actually provides since we no longer use the default here.

Now we want to make the output of the controller available trough a dedicated page type:

api_tag = PAGE
api_tag {
  config {
    disableAllHeaderCode = 1
    debug = 0
    no_cache = 1
    additionalHeaders {
      10 {
        header = Content-Type: application/json
        replace = 1
      }
    }
  }
  typeNum = 1452982642
  10 < tt_content.list.20.jsonexample_json_tag
}

If we include this TypoScript and then access our page with the parameter &type=1452982642 we will get the JSON representation of our tag objects (assumed we at least created one in a sysfolder in the TYPO3 backend). We always get the list action rendered as we specified this as the default action of our plugin. We also asuret hat the header Content-Type: application/json is always set for our page type.

The extbase JSON view will render all simple properties. So we should see the uid, pid and title fields rendered as JSON. That was easy.

But what if we want to render just a single tag? And what if we have a more complex object with relations and stuff?

Calm down, we gonna tackle that, too.

Top

How to access a single object (show action)

To access the show action we could just add the parameters as extbase would expect them. We have to set the action to "show" and specify a single tag by its uid. With domain.tld/?type=1452982642&tx_jsonexample_json_tag[action]=show&tx_jsonexample_json_tag[tag]=1 our controller would pick the show action and render the JSON representation of the tag with uid 1.

But that is not pretty. We can improve the situation by adding a something like a HTTP method dispatcher to our controller. This will come in handy when we implement POST, PUT and DELETE requests as well but for now, where we only use GET it will at least allow us to get rid of the action parameter:

/**
* Resolves and checks the current action method name
*
* @return string Method name of the current action
*/
protected function resolveActionMethodName()
{
  switch ($this->request->getMethod()) {
    case 'HEAD':
    case 'GET':
      $actionName = ($this->request->hasArgument('tag')) ? 'show' : 'list';
      break;
    case 'POST':
    case 'PUT':
    case 'DELETE':
    default:
      $this->throwStatus(400, null, 'Bad Request.');
                                      }
  return $actionName . 'Action';
}

We overwrite the resolveActionMethodName method and use the HTTP method for determining the correct action. If the argument 'tag' is set it resolves to 'showAction'. This means if the parameter tx_jsonexample_json_tag[tag] is present it is assumed that we want to show the single tag instead of the list.

The other methods will just trigger a Bad Request error for now as we wont implement them in this tutorial. So we reduced the necessary parameters to ?type=1452982642&tx_jsonexample_json_tag[tag]=1. But further improvement is possible. With realurl.

Top

Make the URL pretty - realurl

We have to configure our page type and the tx_jsonexample_json_tag[tag] parameter. I prefer to use the 'fixedPostVars' for this to enable my pretty API on a dedicated page (or subdomain). This is how the config could look like:

$TYPO3_CONF_VARS['EXTCONF']['realurl']['_DEFAULT']['fixedPostVars']['api'] = [
        [
            'GETvar'   => 'type',
            'valueMap' => [
                'tag' => 1452982642
            ],
        ],
        [
            'cond'        => [
                'prevValueInList' => '1452982642'
            ],
            'GETvar'      => 'tx_jsonexample_json_tag[tag]',
            'lookUpTable' => [
                'table'       => 'tx_jsonexample_domain_model_tag',
                'id_field'    => 'uid',
                'alias_field' => 'uid'
            ],
            'optional'    => true,
        ]
    ];
$TYPO3_CONF_VARS['EXTCONF']['realurl']['_DEFAULT']['fixedPostVars'][1] = 'api';

This config has the following advantages

  • It is active only on the configured page. in the example it is the page with the id 1. But in a real world example it would be something more sophisticated. However I am kind of OK with hardcoding the ID here since it is very unlikely to change.
  • The parameter tx_jsonexample_json_tag[tag] is only mapped if the pageType matched. This is achieved by the 'cond' configuration, which basically means: "If the pageTye is 1452982642 the next parameter will be mapped to tx_jsonexample_json_tag[tag]".
  • It is extendible. We can easily add more plugins that handle different domain objects in the same way without getting the parameters or URLs mixed up.

In combination with the controler above this configuration allows us to access our list view with domain.tld/tag and a single tag e.g. with domain.tld/tag/1.

That is much better.

So we can turn to more complex domain objects.

Top

More complex objects

Let's have a look at another domain objec example: the post. Let's say a post can have multiple tags, so that we have a many to many relation.

namespace DanielGoerz\JsonExample\Domain\Model;
use TYPO3\CMS\Extbase\DomainObject\AbstractEntity;

class Post extends AbstractEntity
{
    /**
     * @var string
     */
    protected $title = '';

    /**
     * @var \TYPO3\CMS\Extbase\Persistence\ObjectStorage<\DanielGoerz\JsonExample\Domain\Model\Tag>
     */
    protected $tags = null;

    public function __construct()
    {
        $this->tags = new \TYPO3\CMS\Extbase\Persistence\ObjectStorage();
    }

    // Getter and setter
}

If we set up the post API exactly like we did the tag API, we end up with a new pagetype 1452982643, the URL domain.tld/post and some duplicated code in the two controllers. But of course the first thing we developers notice is that the property $tags will not be rendered in the JSON representation of our list of posts. And this is because if a property value of the rendered domain object is an array (or an object itself), it is not included by the default configuration. This means the JSON will only include the uid, the title and the pid of each record that was created in the backend.

This is where the default configuration of the JSON view is no longer sufficient. In fact in my opinion it wasn't even sufficient in the tag example above because we most likely don't want to disclose the pid of our records to the public. Why should we?

So how do we change the configuration? That is easy (as it always is). We use our own JSON view:

namespace DanielGoerz\JsonExample\Mvc\View;
use TYPO3\CMS\Extbase\Mvc\View\JsonView as ExtbaseJsonView;

class JsonView extends ExtbaseJsonView
{
    /**
     * @var array
     */
    protected $configuration = [
        'tags' => [
            '_descendAll' => [
                '_exclude' => ['pid'],
            ]
        ],
        'tag' => [
            '_exclude' => ['pid'],
        ],
        'posts' => [
            '_descendAll' => [
                '_exclude' => ['pid'],
                '_descend' => [
                    'tags' => [
                        '_descendAll' => [
                            '_only' => ['uid']
                        ]
                    ]
                ]
            ]
        ],
        'post' => [
            '_exclude' => ['pid'],
            '_descend' => [
                'tags' => [
                    '_descendAll' => [
                        '_only' => ['uid']
                    ]
                ]
            ]
        ]
    ];
}

So we extend the extbase JSON view and only overwrite the $configuration array that is empty by default. We now may use some weird syntax to configure what properties of a certain model should be rendered. And we even can differentiate between the listAction (plural) and the showAction (singular).

There is not much documentation about this syntax out there. In fact the best documentation that I know of is in the code. The most interesting parts are:

  • The array keys are the value names
  • '_exclude': Array of gettable properties to be excluded
  • '_only': Array of gettable properties to render
  • '_descentAll': If a value is an array or an objectStorage the configuration inside a '_descentAll' will be applied to all containing items.
  • '_descent': A property containing an array, an object or an objectStorage will be rendered with the containing configuration. Therefore - theoretically - infinitely nested structures can be configured.

You can also expose the class name or persistence identifier of the object. One last thing: The configuration respects all publicly available getter methods (using Reflection). So you could easily provide a custom getter to better handle the JSON output. For example the method getTags() in our post class returns an ObjectStorage Instance but we might rather want an array rendered in the JSON. So lets change that for the sake of better understanding the mechanics:

namespace DanielGoerz\JsonExample\Domain\Model;
use TYPO3\CMS\Extbase\DomainObject\AbstractEntity;

class Post extends AbstractEntity
{
    // Same as above

    /**
     * Wrapper for JSON view
     *
     * @return array
     */
    public function getTagsArray()
    {
        return $this->tags->toArray();
    }

}

Now we can use this new method in our JSON view configuration.

namespace DanielGoerz\JsonExample\Mvc\View;
use TYPO3\CMS\Extbase\Mvc\View\JsonView as ExtbaseJsonView;

class JsonView extends ExtbaseJsonView
{
    /**
     * @var array
     */
    protected $configuration = [
        // Same as above
        'posts' => [
            '_descendAll' => [
                '_exclude' => ['pid'],
                '_descend' => [
                    'tagsArray' => [
                        '_descendAll' => [
                            '_only' => ['uid']
                        ]
                    ]
                ]
            ]
        ],
        'post' => [
            '_exclude' => ['pid'],
            '_descend' => [
                'tagsArray' => [
                    '_descendAll' => [
                        '_only' => ['uid']
                    ]
                ]
            ]
        ]
    ];
}

If you always want your ObjectStorages converted to Arrays in your JSON response, you don't have to introduce a converter method for every ObjectStorage property in every model. You can achieve this in your JsonView class by overriding the methof transformValue() (thanks to Sven KĆ¼lpmann for pointing this out to me). here is how that looks like:

class JsonView extends ExtbaseJsonView
{
    /**
     * Always transforming ObjectStorages to Arrays for the JSON view
     *
     * @param mixed $value
     * @param array $configuration
     * @return array
     */
    protected function transformValue($value, array $configuration)
    {
        if ($value instanceof ObjectStorage) {
            $value = $value->toArray();
        }
        return parent::transformValue($value, $configuration);
    }
}

The last thing we have to do is now that we have a configuration we need to tell the view in our controller actions what value from the configuration it should apply. This is done via $this->view->setVariablesToRender().

Let's see how it looks for the TagController:

use DanielGoerz\JsonExample\Domain\Model\Tag;
use TYPO3\CMS\Extbase\Mvc\Controller\ActionController;

class TagController extends ActionController
{
    // Same as above

    /**
     * @return void
     */
    public function listAction()
    {
        $this->view->setVariablesToRender('tags');
        $this->view->assign('tags', $this->tagRepository->findAll());
    }

    /**
     * @param Tag $tag
     * @return void
     */
    public function showAction(Tag $tag)
    {
        $this->view->setVariablesToRender('tag');
        $this->view->assign('tag', $tag);
    }
}

We have a pretty nice API now that returns our posts and tags as JSON either as a list or a single item. The API is accessible via reasonable URLs (keep in mind that the pretty URLs are only possible with a realurl config) and can easily be extended and adapted. It no longer renders the pids of our records and handles our many to many relation in a custom way. You should now be able to setup something like this for your real world application.

The next interesting thing may be to get the POST, PUT and DELETE requests covered. But that's another story. And maybe I tell it ;).

Please have a look at the repository at Github where I removed the code duplication by introducing an AbstractApiController. Following this pattern a new concrete Controller has to implement only a minimum of code to get ready.

If you read until this very last sentence of the post, I'd appreciate your feedback, if you got any.

Top

Further Information