Facebook develops a new, machine learning-powered system enabling users to recreate the style of the text on virtually every photo with just a single word sample, which can lead to new developments in translation, editing, replication, etc.
As you can see here, Facebooks TextStyleBrush project will enable users to replace text in existing images, including background scenes, using only one shot and a single word as reference point. The process is a significant advance over the current text replication models, and can, as noted, have a variety of potential applications.
As explained by Facebook:
“Unlike most AI systems that can do this for well-defined, specialized tasks, TextStyleBrush is the first self-monitored AI model that replaces text in images of both handwriting and scenes – in one shot – using a single sample word. “
Facebook specifically notes that the project can help translate text into images, allowing users to interpret even handwritten characters in different languages (example below), while also enabling users to create personalized messages and captions , together with other applications. .
But it can also be abused. Through this process, people could easily remove watermarks from photos, or they could change elements of real photos to change their meaning. Suppose someone were to use it, for example, to change an old university photo of a political candidate holding a sign, so that he / she apparently promotes an offensive slogan rather than the message?
Facebook is aware of these risks, and this is partly why it released the risks research paper on the project, na “the promotion of additional research and dialogue that prevents deep false text attacks.
“If researchers and practitioners of AI can outsmart the adversaries to build this technology, we can learn to better detect this new style of deepening and build robust systems to combat it.”
Yet it seems risky. Handy, sure, especially if there is a text style you might want to recreate and to develop the translation. But it is possible that the risks outweigh the benefits, at least in terms of a broader public implementation.
Maybe Facebook will then just ever use it in translation tools, but it seems like it might have additional value that Facebook would like to capitalize on.
Either way, this is an interesting project – you can read more about the TextStyleBrush project here.