TYPO3 caching framework in your own extension

Here is a situation I have seen more than once. An extension shows data from an external service, for example opening hours or events from a city API. The plugin is USER_INT because “the data must be fresh”, and every page view makes an HTTP call. The external service answers in one or two seconds on a good day. On a bad day the whole page waits.

The fix is not to make the page cached and hope. The fix is to cache the expensive part, the API answer, with the TYPO3 caching framework. It has been in the core since 4.3, and it is still one of the most underused APIs in extensions I review.

Step 1: register the cache

A cache is a frontend (how values are stored: VariableFrontend for any serializable PHP value, PhpFrontend for PHP code) plus a backend (where they are stored). Register it in ext_localconf.php:

<?php

defined('TYPO3') or die();

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['myext_feed'] ??= [];
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['myext_feed']['groups'] ??= ['pages'];

That’s really all. When frontend and backend are not set, the CacheManager uses VariableFrontend with Typo3DatabaseBackend. The ??= is important: it lets the project override the configuration in config/system/additional.php without fighting your extension.

The groups entry decides which “flush” actions clear your cache. Without it, the default group is all. I put this cache in pages, because its content is shown on pages and should be cleared together with the page cache. You will see in step 4 why this choice brings a nice bonus.

With the database backend, TYPO3 needs two tables, cache_myext_feed and cache_myext_feed_tags. The Database Analyzer in the Install Tool or ddev typo3 extension:setup creates them for you.

Step 2: inject the cache

Don’t call GeneralUtility::makeInstance(CacheManager::class)->getCache() all over the place. Define the cache as a service and inject it, just like the core does for cache.hash or cache.pages:

# Configuration/Services.yaml
services:
  _defaults:
    autowire: true
    autoconfigure: true
    public: false

  VendorMyExt:
    resource: '../Classes/*'

  cache.myext_feed:
    class: TYPO3CMSCoreCacheFrontendFrontendInterface
    factory: ['@TYPO3CMSCoreCacheCacheManager', 'getCache']
    arguments: ['myext_feed']

  VendorMyExtServiceFeedService:
    arguments:
      $cache: '@cache.myext_feed'

Instead of the last three lines you can also write #[Autowire(service: 'cache.myext_feed')] in front of the constructor parameter. Several core classes do exactly that with cache.runtime.

Step 3: read, fetch, tag, store

<?php

declare(strict_types=1);

namespace VendorMyExtService;

use TYPO3CMSCoreCacheFrontendFrontendInterface;
use TYPO3CMSCoreHttpRequestFactory;

final readonly class FeedService
{
    public function __construct(
        private FrontendInterface $cache,
        private RequestFactory $requestFactory,
    ) {}

    /**
     * @param array{uid: int, url: string, lifetime: int} $feed record from tx_myext_feed
     */
    public function getItems(array $feed): array
    {
        $identifier = 'feed_' . $feed['uid'] . '_' . sha1($feed['url']);
        $items = $this->cache->get($identifier);
        if (is_array($items)) {
            return $items;
        }

        try {
            $response = $this->requestFactory->request($feed['url'], 'GET', ['timeout' => 3]);
            $items = json_decode((string)$response->getBody(), true, 32, JSON_THROW_ON_ERROR);
        } catch (Throwable) {
            // Remember the failure for one minute, do not hammer a broken API
            $this->cache->set($identifier, [], ['tx_myext_feed_' . $feed['uid']], 60);
            return [];
        }

        $this->cache->set(
            $identifier,
            $items,
            ['tx_myext_feed', 'tx_myext_feed_' . $feed['uid']],
            $feed['lifetime'] ?: 3600
        );
        return $items;
    }
}

A few things that I learned the hard way:

Cache identifiers allow only letters, numbers and _ - % &, at most 250 characters. A URL with slashes and colons is not a valid identifier, so hash it. isValidEntryIdentifier() tells you if you are unsure.
get() returns false for a miss. If you store a value that can be false itself, wrap it in an array, otherwise you cannot tell the difference.
Cache the failure too, with a short lifetime. Otherwise a broken API makes every request wait for the timeout, which was the original problem.
Always set a timeout on outgoing requests. RequestFactory passes it to Guzzle.

Now the plugin itself can be a normal cached plugin again, or stay USER_INT but cheap. Both are fine, the slow part is gone.

Step 4: flush by tag at the right moment

Tags are the real power. Here the bonus from step 1 comes in. When an editor saves a record, DataHandler collects the tags <table> and <table>_<uid> (for example tx_myext_feed_12) and flushes them in all caches of the group pages. Because my cache is in that group and I used the same tag names, editing the feed record in the backend empties exactly the entries of that feed. No hook, no event listener, no code at all. This holds as long as nobody disabled it with TCEMAIN.clearCache_disable in page TSconfig.

For other moments there are more tools:

# TYPO3 v14: flush by tag from the command line, e.g. in a cron job after the API publishes new data
ddev typo3 cache:flushtags tx_myext_feed -g pages

# Page TSconfig: saving anything on this page also flushes all feed entries
TCEMAIN.clearCacheCmd = cacheTag:tx_myext_feed

The cache:flushtags command is new in v14 (Feature-104054). In v13 you would write a tiny command yourself that calls $this->cache->flushByTag('tx_myext_feed'). And when you really need code on save, $cache->flushByTag() or CacheManager::flushCachesInGroupByTags() are the calls to use.

Step 5: choose the backend for production

The database backend is a good default. It works everywhere and survives deployments. For production I decide per cache:

Few, big entries, read on many pages: database backend is fine. Keep defaultLifetime sensible, or the table grows forever. Garbage collection runs with the scheduler task “Caching framework garbage collection”.
Many small entries and lots of traffic: RedisBackend. It supports tags, which SimpleFileBackend does not.
Data needed only during one request: TransientMemoryBackend, or simply the core cache.runtime.

Since the extension registered only defaults with ??=, the project decides in config/system/additional.php:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['myext_feed']['backend']
    = TYPO3CMSCoreCacheBackendRedisBackend::class;
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['myext_feed']['options'] = [
    'hostname' => 'redis',
    'port' => 6379,
    'database' => 3,
    'defaultLifetime' => 3600,
];

Give each cache its own Redis database number, so a flush of one never wipes another. One note for v14: if you wrote your own cache backend, the interfaces are now strictly typed and AbstractBackend lost the old constructor argument (Breaking-107315). Using the core backends as shown above is not affected.

If this post saved one of your pages from a slow API, I will be really happy. Cheers!

Using the TYPO3 caching framework in your own extension

Step 1: register the cache

Step 2: inject the cache

Step 3: read, fetch, tag, store

Step 4: flush by tag at the right moment

Step 5: choose the backend for production

Further reading

Leave a comment Cancel reply