Solve zsh cursor positioned at the beginning in zsh history lookup using arrow key for Ubuntu

Zsh under Ubuntu has a strange behaviour. When you have typed some commands and use up arrow key to look back, the cursor is always at the beginning of the line. This can be solved very easily.