New video colorization developments: The coronation of Queen Wilhelmina (1898). Goodbye to DeOldify?

The purpose of this film is to show you the latest developments with colorizing historic B&W film footage in conjunction with interesting footage about Dutch Queen Wilhelmina's corronation in Amsterdam on the 6th of September 1898.
Back in 2018 young American Jason Antic surprised the whole world with his great photo and film colorizer software called DeOldify that is based on Artificial Intelligence.
Especially people who are interested in history got very excited about the new possibilities to colorize old footage in order to show that history never was in black and white and to bring it closer to home, especially to entice young audiences.
At the time Jason was very kind to release his brainchild for free to the world by enabling technically oriented people to use his software on line via Github Colab.
That was four years ago. Eversince the entire community of film colorizers has been waiting for the release of an improved version of the DeOldify video colorizer. I most certainly was one of them. The sad thing is that since 2018 the promisses and ideology behind the original intent to open DeOldify up to the entire world have not materialized. In stead the development of a better video colorizer has come to a halt and DeOldify's great photo colorizer has been hidden behind a huge MyHeritage paywall. In order to use it, one has to take a subscription on MyHeritage's most expensive package at 259 US$ a year, regardless or not whether one is interested in geneology!
Over the past years, DeOldify has announced, mainly via Twitter that they were working on a new video colorizer. On a number of occasions everybody's appetite was wet by seeing what great progress was being made in their offices. But time after time the enthousiasme was dampened by statements that it was not ready yet for release and that the
commercial issues needed to be sorted out.
So after 4 years that is where we still stand today. I still have no idea if and when a photo and video colorizer, preferably for use on Windows PC's with a Nvidia Graphics card, will ever materialize, despite the fact that to my opinion it could become a huge success and a direct competitor to companies like Corel and Adobe Photoshop.
Fortunately, DeOldify is no longer the only game in town. One project that I recently found, has huge potential, especially because the colorization is based on properly colorized reference frames and not an A.I. that has to best guess the colours on the basis of object shapes and grey tones in the B&W source footage. This colorized is based on the work of Richard Zhang, Phillip Isola, Alexei A. Efros et al.
While listening to my previous rant you have been watching how a not very high quality B&W film about the coronation of Queen Wilhelmina in 1898 can be meticulously restored and subsequently colorized by means of the 2018 DeOldify Colab version. Here is what DeOldify can achieve.
The results are not bad but nevertheless there is a lot to be still desired.
For example the many flags are not colorized at all and most of the image is a collection of various shades of beige, brown and black.
So I set out to manually colorize a reference frame in order to try out the new colorizer A.I. that I already mentioned. Here you can see what I did: flags were given the correct red, white and blue color, the roof of the station was made more red-ish, the latern post was colored green and many robes, signs and objects were re-colored.
Subsequently this reference frame was fed to the new colorizer and here are the stunning results!
As you probably saw there are still elements that could be further improved. For example the Dutch flag no longer correctly colored when a gust of wind made it fold over, and also the coats of drivers of the carriages did not stay red until the end. But overall the result is much better than with DeOldify.
The great news for historians is that they can now drop their criticism that automatic colorizers make huge errors as regards giving object the historic correct color. Flags, uniforms, period clothing, birds, animals etcetera can now given the correct color! It only requires the person who colorizes the reference frame to do his or her homework, in order to select the right colors and ofcourse be skillful at their job. Color fluctuations which have plagued DeOldify for years are now also a thing of the past.
The only drawback is that it takes substantial amounts of work to get the best result. Making a really good reference frame can take for one hour to a whole day and this has to be repeated for every separate scene in the film.
So to sum up, this new video colorization technology has huge potential. Hopefully this video will also encourage DeOldify to make haste to release its long overdue new video colorizer that is based on the great technology that is still exclusively hidden in the realms of MyHeritage.
Source: Beeld En Geluid
Music: Trevor Kowalski.