Using Rector with TYPO3

In an article from 2017 I talked about how to upgrade a TYPO3 project to a new major TYPO3 version. Back then we already had the amazing extension scanner in place, that can tell us somewhat reliably that parts of our extension code are affected by breaking changes or core deprecations. We can then check the lines that the extension scanner finds to fix them - or in case of a false positive - tell the scanner to ignore the line during future scans.

With rector (Website, GitHub) we can go one step further and actually let the scanner apply the necessary changes to the code. This can be an enormous help when we need to keep extensions compatible with newer TYPO3 versions.

In this post we will look at rector. What it is, how it works and how we can use it in our projects.

Before we start, a big shout out and many kudos have to go to Sebastian Schreiber (say hi at Twitter) who has started the typo3-rector project at GitHub some time ago and who has implemented so many rectors over the time, that the package has been listed in the rectorphp Readme.md as an framework specific rule set (original Tweet).

Thanks for all your work Sebastian!

Tweet from rectorphp mentioning typo3-rector package has been added to the Readme file — Tweet from rectorphp

Rector in a Nutshell

Rector combines static PHP code parsing with many (sets of) rules of how to change and migrate the code. It also applies the changes to the code. Rector relies on the amazing and popular package nikic/php-parser (GitHub).

In short, the PHP code is parsed into an Abstract Syntax Tree (AST) - basically a representation of the PHP code in node objects. While iterating the AST every node is checked against the rules (aka rectors) and if a node meets the rectors conditions and requirements, the changes are applied to the node. After the iteration is done, the AST is written back to PHP code.

A more detailed explanation of how this works can be found here.

When writing AST back to PHP code, rector might not respect any CGL rules we had applied before. We should therefore have a CGL tool at place as well to make the resulting code pretty again. Refer to my article about the php-cs-fixer in TYPO3 projects if you want to learn how we achieve this with little effort.

Top

Installation and Configuration

Rector has out-of-the-box support for TYPO3. Therefore we only need to require the rector/rector package.

composer require --dev ssch/typo3-rector

typo3-rector has been part of the main package rector/rector for a while but was transferred back to its own repository to ease the development process (read more here). If we install version 0.13.4 of rector/rector it ships the TYPO3 related rules up to version 11.

We will continue under the assumption, that the above command has been used to install ssch/typo3-rector.

After rector is installed, we can execute the commands via CLI:

./vendor/bin/rector

Rector is usually configured in a PHP file. If we do not specify a path to a configuration file with --config when executing commands, rector expects a rector.php file at the project root.

To start us off, rector ships with a prepared rector.php that we can copy to our project:

cp ./vendor/ssch/typo3-rector/templates/rector.php.dist rector.php

Let's have a look at this file:

return static function (RectorConfig $rectorConfig): void {

    // If you want to override the number of spaces for your typoscript files you can define it here, the default value is 4
    // $parameters = $rectorConfig->parameters();
    // $parameters->set(Typo3Option::TYPOSCRIPT_INDENT_SIZE, 2);

    $rectorConfig->sets([
        Typo3LevelSetList::UP_TO_TYPO3_11,
    ]);

    // Define your target version which you want to support
    $rectorConfig->phpVersion(PhpVersion::PHP_74);

    // If you only want to process one/some TYPO3 extension(s), you can specify its path(s) here.
    // If you use the option --config change __DIR__ to getcwd()
    // $rectorConfig->paths([
    //    __DIR__ . '/packages/acme_demo/',
    // ]);

    // When you use rector there are rules that require some more actions like creating UpgradeWizards for outdated TCA types.
    // To fully support you we added some warnings. So watch out for them.

    // If you use importNames(), you should consider excluding some TYPO3 files.
    $rectorConfig->skip([
        // @see https://github.com/sabbelasichon/typo3-rector/issues/2536
        __DIR__ . '/**/Configuration/ExtensionBuilder/*',
        // We skip those directories on purpose as there might be node_modules or similar
        // that include typescript which would result in false positive processing
        __DIR__ . '/**/Resources/**/node_modules/*',
        __DIR__ . '/**/Resources/**/NodeModules/*',
        __DIR__ . '/**/Resources/**/BowerComponents/*',
        __DIR__ . '/**/Resources/**/bower_components/*',
        __DIR__ . '/**/Resources/**/build/*',
        __DIR__ . '/vendor/*',
        __DIR__ . '/Build/*',
        __DIR__ . '/public/*',
        __DIR__ . '/.github/*',
        __DIR__ . '/.Build/*',
        NameImportingPostRector::class => [
            'ext_localconf.php',
            'ext_tables.php',
            'ClassAliasMap.php',
            __DIR__ . '/**/Configuration/*.php',
            __DIR__ . '/**/Configuration/**/*.php',
        ]
    ]);

    // If you have trouble that rector cannot run because some TYPO3 constants are not defined add an additional constants file
    // @see https://github.com/sabbelasichon/typo3-rector/blob/master/typo3.constants.php
    // @see https://github.com/rectorphp/rector/blob/main/docs/static_reflection_and_autoload.md#include-files
    // $parameters->set(Option::BOOTSTRAP_FILES, [
    //    __DIR__ . '/typo3.constants.php'
    // ]);

    // register a single rule
    // $rectorConfig->rule(\Ssch\TYPO3Rector\Rector\v9\v0\InjectAnnotationRector::class);

    /**
     * Useful rule from RectorPHP itself to transform i.e. GeneralUtility::makeInstance('TYPO3\CMS\Core\Log\LogManager')
     * to GeneralUtility::makeInstance(\TYPO3\CMS\Core\Log\LogManager::class) calls.
     * But be warned, sometimes it produces false positives (edge cases), so watch out
     */
    // $rectorConfig->rule(\Rector\Php55\Rector\String_\StringClassNameToClassConstantRector::class);

    // Optional non-php file functionalities:
    // @see https://github.com/sabbelasichon/typo3-rector/blob/main/docs/beyond_php_file_processors.md

    // Rewrite your extbase persistence class mapping from typoscript into php according to official docs.
    // This processor will create a summarized file with all the typoscript rewrites combined into a single file.
    /* $rectorConfig->ruleWithConfiguration(\Ssch\TYPO3Rector\FileProcessor\TypoScript\Rector\v10\v0\ExtbasePersistenceTypoScriptRector::class, [
        \Ssch\TYPO3Rector\FileProcessor\TypoScript\Rector\v10\v0\ExtbasePersistenceTypoScriptRector::FILENAME => __DIR__ . '/packages/acme_demo/Configuration/Extbase/Persistence/Classes.php',
    ]); */
    // Add some general TYPO3 rules
    $rectorConfig->rule(ConvertImplicitVariablesToExplicitGlobalsRector::class);
    $rectorConfig->ruleWithConfiguration(ExtEmConfRector::class, [
        ExtEmConfRector::ADDITIONAL_VALUES_TO_BE_REMOVED => []
    ]);

    // Modernize your TypoScript include statements for files and move from <INCLUDE /> to @import use the FileIncludeToImportStatementVisitor (introduced with TYPO3 9.0)
    // $rectorConfig->rule(\Ssch\TYPO3Rector\FileProcessor\TypoScript\Rector\v9\v0\FileIncludeToImportStatementTypoScriptRector::class);
};

I thought about repeating the contents of the rector.php file (and most importantly its comments) in my own words, but I think going through the file would be of greater benefit, because that is what you would start with when you actually set rector up in a project. So please take the time and go through the file to understand what can be configured here.

Now, that we learned something about the configuration, let's see how to actually use rector and what it does to our code.

Top

How to use Rector

To run rector with our current configuration, we execute the process command.

$ ./vendor/bin/rector process src/extensions --dry-run

 60/60 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

1 file with changes
===================

1) src/extensions/my_ext/Classes/DataProcessing/CategoriesProcessor.php

    ---------- begin diff ----------
--- Original
+++ New
@@ @@

 namespace Vendor\MyExt\DataProcessing;

+use TYPO3\CMS\Core\Database\ConnectionPool;
 /*
  * This file is part of TYPO3 CMS-based project my-project
  *
@@ @@

     private function getQueryBuilder(): QueryBuilder
     {
-        $connectionPool = GeneralUtility::makeInstance(\TYPO3\CMS\Core\Database\ConnectionPool::class);
+        $connectionPool = GeneralUtility::makeInstance(ConnectionPool::class);
         return $connectionPool->getQueryBuilderForTable('sys_category');
     }
    ----------- end diff -----------

 [OK] Rector is done! 1 file would have changed (dry-run).

In this example, we apply the configured options and rule sets to everything under the path src/extensions.

With --dry-run we prevent rector from actually changing any files. Instead, we get a command line output of how many files would be affected by the current set of rules and what the git diffs of the changes would look like. If we apply a new set of rectors for the first time, it might be a good idea to do it as a dry run. The dry run could also be integrated in a code analyzing CI pipeline to assure that new or changed code does no introduce anything that rector would migrate.

As this example only imports a namespace, it is a) pretty boring and b) not TYPO3 specific.

Let's look at an actually TYPO3 rector:

Top

TYPO3 Example

As mentioned above, there are many many great TYPO3 rectors. We will pick one of them and demonstrate its power: the MoveRenderArgumentsToInitializeArgumentsMethodRector. Before we discover that the rector does what its name suggests, this seems like a good opportunity to quickly see how we can execute one single rule with rector.

To achieve this we need to not include the whole set of rules in the rector.php (remove the lines) and register the rule we want to execute alone ourselves:

// rector.php
return static function (RectorConfig $rectorConfig): void {
    $rectorConfig->import(__DIR__ . '/vendor/ssch/typo3-rector/config/config.php');
    $rectorConfig->importNames(false, false);
    $rectorConfig->rule(\Ssch\TYPO3Rector\Rector\v9\v0\MoveRenderArgumentsToInitializeArgumentsMethodRector::class);
}

The ->import() method makes sure that all the dependencies of the rules are registered in the service container. We just point to the typo3-rector config.php file, as everything we need is already there. It also enables importNames, so we disable that again to only run the one rector we want.

With this configuration we can move on to our example.

Imagine this (very questionable) ViewHelper:

namespace DanielGoerz\Basis\ViewHelpers;

use TYPO3Fluid\Fluid\Core\ViewHelper\AbstractViewHelper;

class LegacyViewHelper extends AbstractViewHelper
{
    public function render($parameter, int $number = 5, string $string = 'default')
    {
        return 'Number: ' . $number
            . '. String: ' . $string
            . '. 3rd parameter: ' . ($parameter ?? 'not set');
    }
}

You probably already figured it out by yourself, but I spell it out nevertheless: in this example, we do not care for what the ViewHelper does but only care for it's arguments and how they are registered. Having the arguments of ViewHelpers registered as literal arguments of the render() method has been deprecated way back with TYPO3 9.0 (Deprecation RST).

The MoveRenderArgumentsToInitializeArgumentsMethodRector migrates the code to the correct way of registering ViewHelper arguments: the initializeArguments() method.

If we run rector now we will get the following output:

./vendor/bin/rector process src/extensions

 64/64 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

1 file with changes
===================

1) src/extensions/basis/Classes/ViewHelpers/LegacyViewHelper.php

    ---------- begin diff ----------
--- Original
+++ New
@@ @@

 class LegacyViewHelper extends AbstractViewHelper
 {
-    public function render($parameter, int $number = 5, string $string = 'default')
+    public function render()
     {
+        $parameter = $this->arguments['parameter'];
+        $number = $this->arguments['number'];
+        $string = $this->arguments['string'];
         return 'Number: ' . $number
             . '. String: ' . $string
             . '. 3rd parameter: ' . ($parameter ?? 'not set');
+    }
+
+    public function initializeArguments(): void
+    {
+        parent::initializeArguments();
+        $this->registerArgument('parameter', 'mixed', '', true);
+        $this->registerArgument('number', 'int', '', false, 5);
+        $this->registerArgument('string', 'string', '', false, 'default');
     }
 }
    ----------- end diff -----------


Applied rules:

 * MoveRenderArgumentsToInitializeArgumentsMethodRector (https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.0/Deprecation-81213-RenderMethodArgumentOnViewHelpersDeprecated.html )

 [OK] Rector is done! 1 file has been changed.

Amazing.

Our ViewHelper looks like this now:

namespace DanielGoerz\Basis\ViewHelpers;

use TYPO3Fluid\Fluid\Core\ViewHelper\AbstractViewHelper;

class LegacyViewHelper extends AbstractViewHelper
{
    public function render()
    {
        $parameter = $this->arguments['parameter'];
        $number = $this->arguments['number'];
        $string = $this->arguments['string'];
        return 'Number: ' . $number
            . '. String: ' . $string
            . '. 3rd parameter: ' . ($parameter ?? 'not set');
    }

    public function initializeArguments(): void
    {
        parent::initializeArguments();
        $this->registerArgument('parameter', 'mixed', '', true);
        $this->registerArgument('number', 'int', '', false, 5);
        $this->registerArgument('string', 'string', '', false, 'default');
    }
}

We see the type hinted arguments $number and $string have been migrated accordingly. $parameter could only be identified as mixed type. The default values are correctly set as are the flags for required arguments.

Now imagine having dozens and dozens of ViewHelpers migrated with one single command. And this is only one rule of many. It is easy to see how your upgrade can benefit from typo3-rector.

There are rectors for migrations regarding doctrine, extbase annotations, dependency injection, TCA syntax changes, removed or deprecated TYPO3 API and so much more. We cant possibly list everything here. It might be worth just trying it out and apply ruleset for ruleset to your project to see what changes rector suggests to your current code base.

For an even more detailed introduction please refer to the video recording of Sebastians Session at the TYPO3 Camp Rhein Ruhr 2019:

Click here to load the video from the YouTube no-cookie-Domain. Note that loading this video might enable YouTube to track you nevertheless.

Sebastian Schreiber - Benefits of Rector PHP and how to use it (TYPO3 Camp Rhein Ruhr 2019)

This concludes the article as we now know how to install, configure and use rector with TYPO3. If you use typo3-rector in your projects please consider contributing back by providing additional rectors or bugfixes as pull requests to Sebastians repository, so that we as a community can make rector an essential tool for TYPO3 upgrades and extension code migration.

Thanks again for reading and kudos to Sebastian (again).

Top

Further Information

typo3-rector.com
[german] TYPO3 Extension Update Live Coding mit RectorPHP - Sebastian Schreiber (2021, YouTube Video, Channel: Never Code Alone)
typo3-rector at GitHub

More Articles