You can now become a hero and support this Blog at GitHub Sponsors or Patreon.

More about support

In an article from 2017 I talked about how to upgrade a TYPO3 project to a new major TYPO3 version. Back then we already had the amazing extension scanner in place, that can tell us somewhat reliably that parts of our extension code are affected by breaking changes or core deprecations. We can then check the lines that the extension scanner finds to fix them - or in case of a false positive - tell the scanner to ignore the line during future scans.

With rector (Website, GitHub) we can go one step further and actually let the scanner apply the necessary changes to the code. This can be an enormous help when we need to keep extensions compatible with newer TYPO3 versions.

In this post we will look at rector. What it is, how it works and how we can use it in our projects.

Before we start, a big shout out and many kudos have to go to Sebastian Schreiber (say hi at Twitter) who has started the typo3-rector project at GitHub some time ago and who has implemented so many rectors over the time, that the package has been listed in the rectorphp Readme.md as an framework specific rule set (original Tweet).

Thanks for all your work Sebastian!

Rector in a Nutshell

Rector combines static PHP code parsing with many (sets of) rules of how to change and migrate the code. It also applies the changes to the code. Rector relies on the amazing and popular package nikic/php-parser (GitHub).

In short, the PHP code is parsed into an Abstract Syntax Tree (AST) - basically a representation of the PHP code in node objects. While iterating the AST every node is checked against the rules (aka rectors) and if a node meets the rectors conditions and requirements, the changes are applied to the node. After the iteration is done, the AST is written back to PHP code.

A more detailed explanation of how this works can be found here.

When writing AST back to PHP code, rector might not respect any CGL rules we had applied before. We should therefore have a CGL tool at place as well to make the resulting code pretty again. Refer to my article about the php-cs-fixer in TYPO3 projects if you want to learn how we achieve this with little effort.

Top

Installation and Configuration

Rectoir has out-of-the-box support for TYPO3. Therefore we only need to require the rector/rector package.

composer require --dev rector/rector

This might cause conflicts, due to the dependencies of rector, especially when we try to install it in a not quite up to date TYPO3 project (e.g. 8 LTS). Please refer to the projects' installation.md to find out more about less conflicting ways of installing typo3-rector and even how to use it on non-composer TYPO3 projects.

After rector is installed, we can execute the commands via CLI:

./vendor/bin/rector

Rector is usually configured in a PHP file. If we do not specify a path to a configuration file with --config when executing commands, rector expects a rector.php file at the project root.

To start us off, rector can generate a rector.php containing a basic configuration for TYPO3 projects with the command init --template-type=typo3:

./vendor/bin/rector init --template-type=typo3


 [OK] "rector.php" config file was added

Let's have a look at the generated file:

return static function (ContainerConfigurator $containerConfigurator): void {
    // get parameters
    $parameters = $containerConfigurator->parameters();

    // Define what rule sets will be applied
    $parameters->set(Option::SETS, [
        Typo3SetList::TYPO3_95
    ]);

    // FQN classes are not imported by default. If you don't do it manually after every Rector run, enable it by:
    $parameters->set(Option::AUTO_IMPORT_NAMES, true);

    // this will not import root namespace classes, like \DateTime or \Exception
    $parameters->set(Option::IMPORT_SHORT_CLASSES, false);

    // this will not import classes used in PHP DocBlocks, like in /** @var \Some\Class */
    $parameters->set(Option::IMPORT_DOC_BLOCKS, false);

    // Define your target version which you want to support
    $parameters->set(Option::PHP_VERSION_FEATURES, PhpVersion::PHP_72);

    // If you would like to see the changelog url when a rector is applied
    $parameters->set(Typo3Option::OUTPUT_CHANGELOG, true);

    // If you set option Typo3Option::AUTO_IMPORT_NAMES to true, you should consider excluding some TYPO3 files.
    $parameters->set(Option::SKIP, [
        NameImportingPostRector::class => [
            'ClassAliasMap.php',
            'ext_localconf.php',
            'ext_emconf.php',
            'ext_tables.php',
            __DIR__ . '/**/TCA/*',
            __DIR__ . '/**/Configuration/RequestMiddlewares.php',
            __DIR__ . '/**/Configuration/Commands.php',
            __DIR__ . '/**/Configuration/AjaxRoutes.php',
            __DIR__ . '/**/Configuration/Extbase/Persistence/Classes.php',
        ],
    ]);
};

The general behavior of rector can be configured with some options, using the Option:: syntax. Additionally, it will output applied rectors alongside URLs to corresponding RST file in TYPO3s changelog.

As these options are pretty self-explanatory and also have helpful code comments, I will not spent too much time on them. There are constants for several PHP versions so that rector knows what language features can be or must not be used. We can configure the behavior in terms of FQCNs and annotations. The most interesting configuration options are Option::SETS as well as Option::SKIP.

With  Option::SETS we can specify whole sets of rectors that we want to apply to our code. As we can see here, typo3-rector provides a lot of sets. Notably every major TYPO3 LTS version has two sets assigned. One for TCA related changes and one for all  changes (including the TCA changes). During a TYPO3 upgrade to a new LTS version we can add all rule sets of the corresponding version number and all versions that we are skipping. In fact, it might be a good idea to also run the rectors of the TYPO3 version we are coming from, as it is likely that our code is already affected by deprecations.

With Option::SKIP we can declare files or paths to be excluded when the rectors are applied. Paths can be excluded globally for all rectors or for single rectors explicitly. In the above example, some TYPO3 specific files (e.g. ext_localconf.php and ext_tables.php among others) are excluded from the NameImportingPostRector, because we do not want FQCN imports in those files.

To skip a path for each and every rector, wen can add it to the option array directly:

$parameters->set(Option::SKIP, [
    'src/extensions/**/Tests/*',
]);

Now, that we learned something about the configuration, let's see how to actually use rector and what it does to our code.

Top

How to use Rector

To run rector with our current configuration, we execute the process command.

$ ./vendor/bin/rector process src/extensions --dry-run

 60/60 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

1 file with changes
===================

1) src/extensions/my_ext/Classes/DataProcessing/CategoriesProcessor.php

    ---------- begin diff ----------
--- Original
+++ New
@@ @@

 namespace Vendor\MyExt\DataProcessing;

+use TYPO3\CMS\Core\Database\ConnectionPool;
 /*
  * This file is part of TYPO3 CMS-based project my-project
  *
@@ @@

     private function getQueryBuilder(): QueryBuilder
     {
-        $connectionPool = GeneralUtility::makeInstance(\TYPO3\CMS\Core\Database\ConnectionPool::class);
+        $connectionPool = GeneralUtility::makeInstance(ConnectionPool::class);
         return $connectionPool->getQueryBuilderForTable('sys_category');
     }
    ----------- end diff -----------

 [OK] Rector is done! 1 file would have changed (dry-run).

In this example, we apply the configured options and rule sets to everything under the path src/extensions.

With --dry-run we prevent rector from actually changing any files. Instead, we get a command line output of how many files would be affected by the current set of rules and what the git diffs of the changes would look like. If we apply a new set of rectors for the first time, it might be a good idea to do it as a dry run. The dry run could also be integrated in a code analyzing CI pipeline to assure that new or changed code does no introduce anything that rector would migrate.

As this example only imports a namespace, it is a) pretty boring and b) not TYPO3 specific.

Let's look at an actually TYPO3 rector:

Top

TYPO3 Example

As mentioned above, there are many many great TYPO3 rectors. We will pick one of them and demonstrate its power: the MoveRenderArgumentsToInitializeArgumentsMethodRector. Before we discover that the rector does what its name suggests, this seems like a good opportunity to quickly see how we can execute one single rule with rector.

To achieve this we remove or empty the Option::SETS section and register a single rule instead:

// rector.php

// get services (needed for register a single rule)
$services = $containerConfigurator->services();

// register a single rule
$services->set(MoveRenderArgumentsToInitializeArgumentsMethodRector::class);

With this configuration we can move on to our example.

Imagine this (very questionable) ViewHelper:

namespace DanielGoerz\Basis\ViewHelpers;

use TYPO3Fluid\Fluid\Core\ViewHelper\AbstractViewHelper;

class LegacyViewHelper extends AbstractViewHelper
{
    public function render($parameter, int $number = 5, string $string = 'default')
    {
        return 'Number: ' . $number
            . '. String: ' . $string
            . '. 3rd parameter: ' . ($parameter ?? 'not set');
    }
}

If you did not already get it: we do not care for what the ViewHelper does but rather it's arguments. Having the arguments of ViewHelpers registered as arguments of the render() method, has been deprecated with TYPO3 9.0 (Deprecation RST).

The MoveRenderArgumentsToInitializeArgumentsMethodRector migrates this structure to the correct way of registering ViewHelper arguments: the initializeArguments() method.

If we run rector now we will get the following output:

./vendor/bin/rector process src/extensions

 64/64 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

1 file with changes
===================

1) src/extensions/basis/Classes/ViewHelpers/LegacyViewHelper.php

    ---------- begin diff ----------
--- Original
+++ New
@@ @@

 class LegacyViewHelper extends AbstractViewHelper
 {
-    public function render($parameter, int $number = 5, string $string = 'default')
+    public function render()
     {
+        $parameter = $this->arguments['parameter'];
+        $number = $this->arguments['number'];
+        $string = $this->arguments['string'];
         return 'Number: ' . $number
             . '. String: ' . $string
             . '. 3rd parameter: ' . ($parameter ?? 'not set');
+    }
+
+    public function initializeArguments(): void
+    {
+        parent::initializeArguments();
+        $this->registerArgument('parameter', 'mixed', '', true);
+        $this->registerArgument('number', 'int', '', false, 5);
+        $this->registerArgument('string', 'string', '', false, 'default');
     }
 }
    ----------- end diff -----------


Applied rules:

 * Ssch\TYPO3Rector\Rector\v9\v0\MoveRenderArgumentsToInitializeArgumentsMethodRector (https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.0/Deprecation-81213-RenderMethodArgumentOnViewHelpersDeprecated.html )

 [OK] Rector is done! 1 file has been changed.

Amazing.

Our ViewHelper looks like this now:

namespace DanielGoerz\Basis\ViewHelpers;

use TYPO3Fluid\Fluid\Core\ViewHelper\AbstractViewHelper;

class LegacyViewHelper extends AbstractViewHelper
{
    public function render()
    {
        $parameter = $this->arguments['parameter'];
        $number = $this->arguments['number'];
        $string = $this->arguments['string'];
        return 'Number: ' . $number
            . '. String: ' . $string
            . '. 3rd parameter: ' . ($parameter ?? 'not set');
    }

    public function initializeArguments(): void
    {
        parent::initializeArguments();
        $this->registerArgument('parameter', 'mixed', '', true);
        $this->registerArgument('number', 'int', '', false, 5);
        $this->registerArgument('string', 'string', '', false, 'default');
    }
}

We see the type hinted arguments $number and $string have been migrated accordingly. $parameter could only be identified as mixed type. The default values are correctly set as are the flags for required arguments.

Now imagine having dozens and dozens of ViewHelpers migrated with one single command. And this is only one rule of many. It is easy to see how your upgrade can benefit from typo3-rector.

There are rectors for migrations regarding doctrine, extbase annotations, dependency injection, TCA syntax changes, removed or deprecated TYPO3 API and so much more. We cant possibly list everything here. It might be worth just trying it out and apply ruleset for ruleset to your project to see what changes rector suggests to your current code base.

For an even more detailed introduction please refer to the video recording of Sebastians Session at the TYPO3 Camp Rhein Ruhr 2019:

 

This concludes the article as we now know how to install, configure and use rector with TYPO3. If you use typo3-rector in your projects please consider contributing back by providing additional rectors or bugfixes as pull requests to Sebastians repository, so that we as a community can make rector an essential tool for TYPO3 upgrades and extension code migration.

Thanks again for reading and kudos to Sebastian (again).

Top

Further Information