Talking to TYPO3 over CLI

The topic is likely to see some further changes and improvements in future releases of TYPO3. This post focuses on TYPO3 7 LTS but most of the things are valid for 6.2 LTS as well.

CLI stands for Command Line Interface and in combination with TYPO3 it means most of the time that we connect (via SSH) to the webserver where our TYPO3 is running and execute tasks on the command line. This comes in handy during deployment where we want several tasks executed. Such as clearing the cache, updating the database scheme and so on. CLI requests are also used in Cron-jobs to schedule repetetive commands.

The TYPO3 core comes with the dispatcher script typo3/cli_dispatch.phpsh that needs to be executable. If you just call the script you should see something like this:

cd /path/to/typo3/document/root
./typo3/cli_dispatch.phpsh
Oops, an error occurred: This script must have a command as first argument.

Valid keys are:

  extbase
  lowlevel_admin
  lowlevel_cleaner
  lowlevel_refindex
  scheduler

We will now see what's behind these commands. And of course we'll see how we can add our own commands.

As of TYPO3 7 LTS there are two main ways a command can be added to the system. First option is the "oldschool" core way with a CommandLineController. Second and highly recommended option is the more modern extbase CommandController approach.

If you are just interested in how to add a custom command, skip the chapters about the oldschool way and head directly over to the extbase part as this is the best practise as of TYPO3 7 LTS.

Top

"Oldschool" CommandLineController

The lowlevel commands and the scheduler are examples of this approach. First thing we need to know is how to add such a CommandLineController to TYPO3.

Top

Registering a CommandLineController

CommandLineControllers have to be registered in $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['cliKeys']. Let's take the lowlevel_cleaner command as an example: It is registered in the ext_localconf.php of sysext lowlevel.

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['cliKeys']['lowlevel_cleaner'] = array(
    function () {
        $cleanerObj = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance(\TYPO3\CMS\Lowlevel\CleanerCommand::class);
        $cleanerObj->cli_main($_SERVER['argv']);
    },
    '_CLI_lowlevel'
);

This is straight forward. The registered array contains the function that will be called when the lowlevel_cleaner command is executed. Arguments are passed via $_SERVER['argv'] directly to the function. The registered array also contains the name of the backend user that will be used within the command. This is _cli_lowlevel for all lowlevel commands that come with the core.

If the user does not exist in your system you will get the following error:

./typo3/cli_dispatch.phpsh lowlevel_cleaner
Oops, an error occurred: No backend user named "_cli_lowlevel" was found!

If the user does exist, the command can be executed. So let's have a look at some code next.

Top

Writing a CommandLineController

The CleanerCommand class that will be called, extends \TYPO3\CMS\Core\Controller\CommandLineController. In this class we find some basic CLI functionality that we can use in our own Command classes. We can describe any option that we accept as an argument in $this->cli_options and provide helpful output in $this->cli_help. Both will take place in the constructor of our class.

Let's illustrate this with a small fraction of what the CleanerCommand does:

/**
 * Constructor
 */
public function __construct()
{
    // Running parent class constructor
    parent::__construct();
    // Adding options to help archive:
    $this->cli_options[] = array('-r', 'Execute this tool, otherwise help is shown');
    $this->cli_options[] = array('-v level', 'Verbosity level 0-3', 'The value of level can be:
0 = all output
1 = info and greater (default)
2 = warnings and greater
3 = errors');
    // Setting help texts:
    $this->cli_help['name'] = 'lowlevel_cleaner -- Analysis and clean-up tools for TYPO3 installations';
    $this->cli_help['author'] = 'Kasper Skaarhoej, (c) 2006';
}

The output of this would be something like:

./typo3/cli_dispatch.phpsh lowlevel_cleaner
NAME:
    lowlevel_cleaner -- Analysis and clean-up tools for TYPO3 installations

OPTIONS:

-s                      Silent operation, will only output errors and
                        important messages.
--silent                Same as -s
-ss                     Super silent, will not even output errors or
                        important messages.
-r                      Execute this tool, otherwise help is shown
-v level                Verbosity level 0-3
                        The value of level can be:
                          0 = all output
                          1 = info and greater (default)
                          2 = warnings and greater
                          3 = errors

AUTHOR:
    Kasper Skaarhoej, (c) 2006

It depends on the implementation what is actually necessary to run a command. Have a look for yourself what the core ships by default. I'll show one concrete task performed by the lowlevel_cleaner CleanerCommand.

./typo3/cli_dispatch.phpsh lowlevel_cleaner syslog -r
*********************************************
syslog
*********************************************
syslog -- Show entries from syslog

Showing last 25 hour entries from the syslog. More features pending. This
is the most basic and can be useful for nightly check test reports.

---------------------------------------------
[syslog]
 [INFO]
---------------------------------------------
Array
(
    [5925] => #5925 xx.xx.xx.xx.xx 11:35 USER[7]: User daniel logged in from 127.0.0.1 ()
    [5926] => #5926 xx.xx.xx.xx.xx 11:35 USER[7]: User daniel has cleared the cache (cacheCmd=system)
)

So what is the extbase approach then?

Top

Extbase CommandController

The 2nd and highly recommended way to add CLI functionality to TYPO3 is to go with an extbase ComandController. As most parts of extbase the underlying concepts are backported from the Flow framework. There is some official documentation about it. But we will cover the basics here nevertheless.

Top

Registering a CommandController

Classes that make use of the extbase CommandController (by extending it) have to be registered as well but in a slightly different way. Let's see how the sysext extensionmanager registers its  ExtensionCommandController in its ext_localconf.php.

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['extbase']['commandControllers'][] =
    \TYPO3\CMS\Extensionmanager\Command\ExtensionCommandController::class;

So instead of a closure and the name of a backend user, we just add the class name of our CommandController to the ['extbase']['commandControllers'] array in SC_OPTIONS.

Top

Writing a CommandController

A registered class usually resides in Classes/Command/ and it needs to extend \TYPO3\CMS\Extbase\Mvc\Controller\CommandController. If you are familiar with the ActionController in extbase frontend plugin development, you will recognize a similar approach in the CLI context. Every method that is named with the appendix Command will be an executable command. But let's take a look at the ExtensionCommandController mentioned above to illustrate this.

use TYPO3\CMS\Extbase\Mvc\Controller\CommandController;

/**
 * CommandController for working with extension management through CLI/scheduler
 */
class ExtensionCommandController extends CommandController
{
    /**
     * Installs an extension by key
     *
     * The extension files must be present in one of the
     * recognised extension folder paths in TYPO3.
     *
     * @param string $extensionKey
     * @return void
     * @cli
     */
    public function installCommand($extensionKey)
    {
        ...
    }
}

You can always use the help command to see what extbase commands are registered in your system. The core comes with a bunch of registered commands. The output should be somewhat like this:

./typo3/cli_dispatch.phpsh extbase help
Extbase 7.6.0
usage: /Users/daniel/vhosts/core/dev/htdocs/./typo3/cli_dispatch.phpsh ./typo3/cli_dispatch.phpsh extbase <command identifier>

The following commands are currently available:

EXTENSION "EXTBASE":
-------------------------------------------------------------------------------
  help                                     Display help for a command

EXTENSION "EXTENSIONMANAGER":
-------------------------------------------------------------------------------
  extension:install                        Installs an extension by key
  extension:uninstall                      Uninstalls an extension by key
  extension:dumpclassloadinginformation    Updates class loading information.

EXTENSION "LANG":
-------------------------------------------------------------------------------
  language:update                          Update language file for each
                                           extension

See '/Users/daniel/vhosts/core/dev/htdocs/./typo3/cli_dispatch.phpsh ./typo3/cli_dispatch.phpsh extbase help <command identifier>' for more information about a specific command.

Note how the PHPDoc DocBlock of the command method is parsed to generate the help text. This goes even further if the help command is used with a command identifier.

./typo3/cli_dispatch.phpsh extbase help extension:install

Installs an extension by key

COMMAND:
  extensionmanager:extension:install

USAGE:
  /Users/daniel/vhosts/core/dev/htdocs/./typo3/cli_dispatch.phpsh ./typo3/cli_dispatch.phpsh extbase extension:install <extension key>

ARGUMENTS:
  --extension-key

DESCRIPTION:
  The extension files must be present in one of the
  recognised extension folder paths in TYPO3.

There are some things that are good to know when developing a CommandController:

  1. When called via CLI, TYPO3 consider itself to be in the BE context. So if you e.g. load TypoScript with the ConfigurationManager, make sure your settings are stored in module.tx_myext.
  2. You can generate output on the command line by using $this->output() (multiline), $this->outputLine() (single line) or $this->outputFormatted() (respects max width, and can have a left padding).
  3. Like in every class initialized by extbase the initializeObject() method will be called upon initialization and before any command.

To execute a command we use the command identifier (usually the name of the CommandController and the command name separeted by a colon). So to execute the installCommand() from the ExtensionCommandController we have to use the command identifier extension:install:

./typo3/cli_dispatch.phpsh extbase extension:install scheduler

An extbase command controller can be executet with admin priviledges as well. If you need to perform some API calls that check for the admin role, you can set the class variable $requestAdminPermissions to true in your commnd controller.

/**
 * @var bool
 */
protected $requestAdminPermissions = true;

Now that we have seen both ways of adding commands to TYPO3 it is time to talk about what is the better approach. And there is a simple answer to that question: the extbase CommandController are preferable over the oldschool API. And yes. There are reasons.

Top

Advantages of the extbase approach

  1. Your commands are automatically available as scheduler tasks.
    With TYPO3 7 LTS a feature was introduced that let you annotate a command with @cli to exclude it from the available commands in the scheduler and make it a CLI only command (Feature RST).
    So the installCommand is annotated with @cli and thus won't be available as scheduler task, because it would not make much sense to install or uninstall extensions on a Cron base. But for your own scripts it could make perfect sense. A frequently appearing usecase is a data import that needs to run e.g. every night.

  2. You can make use of the extbase dependecy injection.
    Since we are in extbase context the @inject annotations as well as the dedicated inject methods are respected.

  3. You can make use of Symfony Console.
    With TYPO3 7 LTS the famous Symfony Console component has been integrated in extbase CommandControllers (Feature RST). It is now possible to interact with the user by e.g. asking for input or leting the user choosing from different options. I copy the example from the RST to illustrate what is possible with this:
namespace Acme\Demo\Command;

use TYPO3\CMS\Extbase\Mvc\Controller\CommandController;

/**
 * My command
 */
class MyCommandController extends CommandController {

        /**
         * @return string
         */
        public function myCommand() {
                // render a table
                $this->output->outputTable(array(
                        array('Bob', 34, 'm'),
                        array('Sally', 21, 'f'),
                        array('Blake', 56, 'm')
                ),
                array('Name', 'Age', 'Gender'));

                // select
                $colors = array('red', 'blue', 'yellow');
                $selectedColorIndex = $this->output->select('Please select one color', $colors, 'red');
                $this->outputLine('You choose the color %s.', array($colors[$selectedColorIndex]));

                // ask
                $name = $this->output->ask('What is your name?' . PHP_EOL, 'Bob', array('Bob', 'Sally', 'Blake'));
                $this->outputLine('Hello %s.', array($name));

                // prompt
                $likesDogs = $this->output->askConfirmation('Do you like dogs?');
                if ($likesDogs) {
                        $this->outputLine('You do like dogs!');
                }

                // progress
                $this->output->progressStart(600);
                for ($i = 0; $i < 300; $i ++) {
                        $this->output->progressAdvance();
                        usleep(5000);
                }
                $this->output->progressFinish();

        }
}

In fact the extension:install command will ask us what extension should be installed if we don't specify an extension key:

./typo3/cli_dispatch.phpsh extbase extension:install
Please specify the required argument "--extension-key":

Have alook at the post TYPO3 v8 - Already awesome if you are interested where this is going in the development of TYPO3 8 LTS.

And the good news just don't stop. You don't have to reinvent the wheel for mere TYPO3 tasks. There are already tools out there.

Top

Shiny extensions

So, now that we know how to add CommandController and how to execute them, I want to mention two extensions that provide a variety of commands that are very usefull during deployment or for maintaining a TYPO3 system.

Top

coreapi

The extension coreapi (Github, TER) from Tobias Liebig provides all kinds of tasks:

EXTENSION "CORE_A_P_I":
-------------------------------------------------------------------------------
  backendapi:lock                          Locks backend access for all users
                                           by writing a lock file that is
                                           checked when the backend is
                                           accessed.
  backendapi:unlock                        Unlocks the backend access by
                                           deleting the lock file

  databaseapi:databasecompare              Database compare.

  cacheapi:clearallcaches                  Clear all caches.
  cacheapi:clearsystemcache                Clear system cache.
  cacheapi:clearallactiveopcodecache       Clears the opcode cache.
  cacheapi:clearconfigurationcache         Clear configuration cache
                                           (temp_CACHED_..).
  cacheapi:clearpagecache                  Clear page cache.
  cacheapi:clearallexceptpagecache         Clear all caches except the page
                                           cache.

  siteapi:info                             Basic information about the system.
  siteapi:createsysnews                    Sys news record is displayed at the
                                           login page.

  extensionapi:info                        Information about an extension.
  extensionapi:listinstalled               List all installed extensions.
  extensionapi:updatelist                  Update list.
  extensionapi:install                     Install(activate) an extension.
  extensionapi:uninstall                   UnInstall(deactivate) an extension.
  extensionapi:configure                   Configure an extension.
  extensionapi:fetch                       Fetch an extension from TER.
  extensionapi:listmirrors                 Lists the possible mirrors
  extensionapi:import                      Import extension from file.

The extension ships a bunch of extbase CommandControllers that provide nearly every administrative action one could perform in the backend as a CLI command. But even this extension got itself already a successor ...

Top

typo3_console

With typo3_console (Github, TER) Helmut Hummel went one step further (as he usually does) and not only provides most tasks the coreapi provides (and adds more on top) but also offers a homogeneous way of executing commands that does no longer need the dispatcher script of TYPO3.

I won't go into much detail here because Helmut wrote it already all by himself. So please check out these links:

A call to a CLI command with an installed typo3_console looks like this:

./typo3cms cache:flush

The list of commands is quite similar to the coreapi:

EXTENSION "TYPO3_CONSOLE":
-------------------------------------------------------------------------------
  cache:flush                              Flushes all caches.
  cache:flushgroups                        Flushes all caches in specified
                                           groups.
  cache:flushtags                          Flushes caches by tags, optionally
                                           only caches in specified groups.
  cache:warmup                             Warmup essential caches such as
                                           class and core caches
  cache:listgroups                         Lists all registered cache groups.

  backend:lock                             Locks backend access for all users
                                           by writing a lock file that is
                                           checked when the backend is
                                           accessed.
  backend:unlock                           Unlocks the backend access by
                                           deleting the lock file

  scheduler:run                            Executes tasks that are registered
                                           in the scheduler module

  cleanup:updatereferenceindex             Updates reference index to ensure
                                           data integrity

  documentation:generatexsd                Generate Fluid ViewHelper XSD Schema

  install:setup                            Alpha version of a setup command.
                                           Use with care and at your own risk!
  install:generatepackagestates            Activates all packages that are
                                           configured in root composer.json or
                                           are required

  database:updateschema                    Update database schema

  configuration:removebypath               Removing system configuration by
                                           path

  frontend:request                         Submit a frontend request

Thank you for reading until the last sentence. The topic is likely to see some further changes and improvements in future releases of TYPO3. This post focused on TYPO3 7 LTS but most of the things are valid for 6.2 LTS as well.

If you want to give me feedback or want to correct an error (that is likely to be there) please contact me via email or on slack.

Have fun on the command line!

Top