Took me some time to figure this command out:

php shell/varnishadm.php ban req.url "~" "."

We recently updated a Magento shop to latest version Magento 2.3.4. First, I was wondering that order confirmation mails were sent as attachments. Meaning there was no content anymore, everything was placed in a file called attachment.html.

After analysing this issue, I stumpled upon this issue. Reason was a newly introduced $part->setDisposition(Mime::DISPOSITION_INLINE);. Thankfully, this could be patched by applying this commit.

If you are trying to send an E-Mail in a cronjob or a console command and encounter following error that the required parameter theme_dir was not passed like this:

[InvalidArgumentException]
Required parameter 'theme_dir' was not passed

Try to wrap your code inside an emulateAreaCode call, which fixes this issue.

// @var $appState Magento\Framework\App\State
$appState->emulateAreaCode('frontend', function () {
    // stuff
});

For a Magento2 shop in production mode, a Varnish cache is essential. I don’t want to dive into details on how to set up your environment with Varnish. This post is about an issue with static assets like stylesheets, javascripts or images. Having a deployment tool like Magallanes or Capistrano in place to deploy your shop, you usually want the cache the be cleared to deliver all changes instantly. One way is to automatically execute bin/magento cache:clean on your server. However, this will not affect static assets. Read on to see why.

Preface

Actually, Magento2 is not to blame here. This issue is combination of PHP7 + less compiler + preg_match + JIT compiler + long base64 encoded strings. But read on, if you are interested.

The issue

This one caused a lot of headache today. Suddenly my locally installed Magento2 Shop had no styles anymore. Well not exactly suddenly, I were trying to port my dev environment to new docker containers and used php:7.0.14-apache as a base image. Diving into this problem was really weird because css files were not correctly built. The less compiler ran into a compilation error. Even using bin/magento setup:static-content:deploy did not help. I got an error like this and couldn’t really figure out what was wrong here.

Compilation from source:
frontend/theme/folder/de_DE/css/styles-m.less
  in _theme.less on line 129, column 1
127 |   url("http://php.net/images/logo.php") format("svg");
128 | }
129 | @font-face {

I tracked this issue down to less statements which where using base64 encoded images like this:

Preface

One thing I like about Magento2 is, that it is completly based on composer. It means you are able to separate development of Magento2 modules from the actual store you are testing it on. This is especially useful, if you are having multiple modules installed in multiple stores.

Preface

This is a status report of my experience trying to get a Magento 2.0.x shop into production mode. I was fighting quite a while to deploy one of our recent shop projects to live production mode. Sounds very simple and straight-forward on the first glance, if you’ve read the docs and have some familarity with the Magento 2 system. However, it was very painful and caused me a lot of headache. I want to share some of the pain I had and maybe help others also confronted with this daunting task.

Lets start with some theory. In Magento 2, you have some folders where the system generates files. In development mode, this is done on the fly and you don’t experience any problems, except that it is fucking slow on initial requests. These folders are:

• var/di
• var/generation
• pub/static/frontend
• pub/static/_cache
• pub/static/adminhmtl
• pub/static/_requirejs

It sounds pretty reasonable that this on the fly generation is not a good idea for production systems, where performance matters. For this case, Magento provides some tasks to generate these files. These tasks are:

• bin/magento setup:di:compile
• bin/magento setup:static-content:deploy

This is, where most of my painful experience began. Not to mention that especially setup:static-content:deploy eats about 30 minutes of your time (everytime) for generating assets for more than 1 store language. I dont’t understand why it needs to take so long. Maybe the process can be improved, I don’t care for now. Waiting time is just annoying everytime.

However on my way, I discovered some core issues in 2.0.x, which finally forced us to update to Magento 2.1.0. So lets get started with a list of detailed issues, I ran into.

Switching to HTTPS

This is not a bug, rather a topic not pretty well documented. For live systems you usually want to have https in place. In theory, pretty simple. Install certificate, configure your web server and set web/secure/base_url. However, as we have varnish and nginx and multiple backend servers in place, I couldn’t persuade them to propagate the correct http headers to Magento. I didn’t want to put more time into debugging this. A workaround for forcing Magento to https internally, is to modify pub/index.php and prepend some variables:

<?php
$_SERVER['HTTPS'] = 'on';
$_SERVER['HTTP_X_FORWARDED_PROTO'] = 'https';
$_SERVER['HTTP_X_FORWARDED_PORT'] = '443';

This is required to force magento to link all assets with https, because otherwise your stylesheets and javascripts would get linked with http, causing security warnings in the browser. However, this is only half of the story. Also setup:static-content:deploy task is affected by this change. In https-mode magento links to pub/static/_requirejs/frontend/Magento/luma/de_DE/secure/requirejs-config.js, which had been pub/static/_requirejs/frontend/Magento/luma/de_DE/requirejs-config.js previously. See the difference? Only Jesus knows, why Magento uses a different requirejs-config.js in a subfolder called secure in this case. I wasn’t able to get this file out of the deploy task, until I discovered somewhere on google, that you need to set the environment variable HTTPS as well. This sucks very hard and I don’t understand why the deploy task can’t just generate everything you need. Long story short, just call your generation tasks with HTTPS="on":

HTTPS="on" php bin/magento setup:static-content:deploy de_DE en_US en_GB

Deploy task generates requirejs-config only for default language

I had a lot of 404 loading errors when switching to another language, which were caused by a missing requirejs-config.js. It turned out, that this file is only generated for the default language. I could work around this by creating some symlinks and this issue got fixed by Magento 2.1. Here is the respective Issue on Github.

Compile task cannot handle preference and plugin for the same class

This one caused a lot of trouble and finally forced me to update to Magento 2.1. Prior versions are obviously not able to handle both preference class and plugin for the same class. It was very very painful to track this issue down, especially because it only happens if you explicilty use bin/magento setup:di:compile, which you don’t do locally in development mode. Thanks for this wasted time. Let me provide some explanation. In Magento 2 you have the option to change behavior of core classes either by preference or by plugin. Both is done in your modules etc/di.xml file. The preference tag basically changes the requested class to your changed one:

<preference for="Magento\Catalog\Model\Product" type="Foo\MyModule\Model\Product" />

Whenever the ObjectManager is called to get an instance of Magento\Catalog\Model\Product it will give you an instance of Foo\MyModule\Model\Product.

With plugins, it is possible to only exchange certain functions. I don’t want to dive too deeply into this topic. For this issue, it is important to know, that magic which is making the plugin system work happens inside of var/di. Magento generates Interceptor classes which forward method calls to your plugin.

<type name="{ObservedType}">
    <plugin name="{pluginName}" type="{PluginClassName}" sortOrder="1" disabled="true"/>
</type>

In development mode, these required Intercepter classes are generated on the fly and everything works perfectly well. But when using bin/magento setup:di:compile, which is expected to generate every necessary Interceptor class inside var/di, some unexpected behavoir might occur. You might expect this generation to be totally transparent. It shouldn’t matter if it is done on the fly or pre-generated, right? Got ya, made this assumption without your dear friend Magento 2. The reason for this is a core bug in releases below 2.1. Before 2.1, pre-generated Interceptor classes do not inherit from classes defined in preference tag. Instead they just inherit from its base class leaving your defined preference totally useless behind, if some plugin hooks into its methods. Here is the respective Issue on Github.

MailCatcher is a nice tool to catch generated E-Mails. However, the default installation is meant for standard localhost set up. When usign Vagrant, Docker or both for isolating different projects it would be nice to use a single installation of MailCatcher on your development machine. This example covers a set up for PHP projects. Usually, when setting up MailCatcher with PHP it is required to configure sendmail_path in your php.ini like this:

sendmail_path = /usr/bin/env catchmail -f some@from.address

The drawback with this approach is, that catchmail is coming from the MailCatcher installation and therefore it would be necessary to install MailCatcher on the virtual machine provided by Vagrant or inside the corresponding docker container. I was however looking for a solution to avoid this overhead and use a single installation on my host system. It is a little bit tricky because the default sendmail is not able to forward mails to another host/port without a lot of configuration overhead. With mini_sendmail however, I found a working solution with following php.ini configuration:

sendmail_path = /usr/bin/mini_sendmail -s192.168.56.1 -p1025 -t -i

From inside a virtual machine, 192.168.56.1 would be the address of the host system and port 1025 the port where CatchMail is listening. It is also required to define the address where CatchMail should listen because the default value is localhost and a connection would not be possible. This is why CatchMail needs to be started with the argument smtp-ip:

catchmail --smtp-ip 192.168.0.1

Inside Docker

However, I had one issue using mini_sendmail inside a docker container. It failed with:

can't determine username

I currently don’t know the reason for this. It must have something to do with the glibc method getLogin() used by mini_sendmail which was not available inside my docker container for whatever reason. However, a dirty workaround was to change the line

username = getlogin();

to

username = "root";

and recompile mini_sendmail which did the trick.

This is a short explanation on how templates of Magento2 can be customized. Making changes to the original template files is bad practise and should be avoided. So there are basically two ways to change a template.

Custom Theme

Inside a custom theme any template can be changed, following the folder hierachy of Magento2. E.g. changing the login.phtml (coming from the module Magento_Customer) filepath would look like this:

app/design/frontend/${VENDORNAME}/${THEMENAME}/Magento_Customer/templates/form/login.phtml

The original source file is living in:

Trying to access an ESI block in a Magento shop may result in following error: Error 403 External ESI requests are not allowed. Makes sense, but for some debugging I needed to request the block manually. Following example using curl applies for a setup where varnish is running on port 80 and apache on 8080.

curl -H "X-Turpentine-Secret-Handshake:1" -H "X-Varnish-Esi-Method: esi" --header 'Host: www.example.de' http://localhost:8080/index.php/turpentine/esi/getBlock/method/esi/access/private/hmac/cf292cebffc75d1521ebfb6784eff77c729418ff6fc0171037b9eabe9862c28d/data/6AryhgsBOYGddSdbsuY3tfaKeRenzd5PP8TyaLFSegp4SJA6wHRhBWmtOWz6HW0xguJUrVdg0he4DBYbnvWPo.W.y88zGfIkkq6KgkkTXOWLemU2j9ZevdGZCGmp6FppZe2LzbvEM5VaAfKunkS3Bo7MV1TL-dI3pqZwrUqToalc-Kboy8NDpTG-9Dc3c7RLISvPquR3eAvKFNCYhX6.ZiO1KH4vGi8nfO1zmFcD9LrFCwQeOIW5tWzYOJXS368fycQjW-Y9Tgk4oNJHZF4Gb59KJTSU6GQM79Vagscrtr2c8MwlQGdOzVcFm1LGsIEA6rMXMHGQBJgVFo5uxZZOVH1yC4zntgZv/