A well-designed desktop or mobile website doesn’t just appear overnight. In fact, you need to go through a long process before you can publish your website. From concept sketches to a production-ready web app can, the entire process can take many months. But what if there were an easier, faster, and more efficient way to code and test a website layout? Thanks to machine learning, specifically deep learning, that dream is now a reality. Experts predict that machine learning and AI technology will change the process of front-end design for web designers in just a few short years. Of course, the question then arises: what will happen to front-end developers once machine learning becomes the norm for web programming?
Deep Learning and Website Design
Deep neural networks are a subclass of machine learning models that can be trained to make inference from large datasets. A neural network learns to do this by analyzing both the data input and the corresponding output. These models are widely used to solve AI challenges such as computer vision and natural language processing. Similar models can be used to design interface elements from relevant data.
There are at least two types of data to train a neural network to learn front-end development. The first is GUI screenshots (pix2code). Which is basically the screen you’re looking at right now. And the second is hand-drawn sketches (sketch2code). Each screenshot and sketch would include its respective code. Once the network has analyzed the initial image, it then moves on to language modeling or figuring out how to code the image in HTML or CSS. In the case of hand-drawn sketches, any written material also goes through a text recognition step.
Once these steps are completed, the neural network can apply its knowledge of objects, object positions, and layouts to code basic front-end elements. Ultimately, the neural network will become capable of “understanding” the visual design at a more abstract level (i.e. the intent behind the code). Deep learning eats up a lot of computing power, but the results are profound – breaking down the barriers to design and streamlining the entire process.
With the millions of websites on the internet, there’s no shortage of training data available for GUI screenshots. And each day thousands of sites go live, ensuring that any deep neural network using this method has access to the most up-to-date design information. This advantage also extends to iOS and Android mobile design capabilities.
Use Cases
AI-based UX design is more than just the latest futurist prediction. Startups and brand names alike are already using the technology in their day-to-day operations. Copenhagen-based startup Uizard has even built an entire business around the concept of transforming images into front-end code for mobile and web apps. The company is the first to build a machine learning algorithm that reads a single UI image and designs platform-specific code. Uizard can turn hand-drawn wireframes and digital designs into elegant, customizable and production-ready code. It reduces the amount of time that it takes for companies, both big and small, to go from concept to a finished product.
But they’re not the only ones looking ahead to the future of front-end design. Airbnb, one of the most recognizable brands in tech, is also getting in on the action. Starting from the premise that “the time required to test an idea should be zero,” the company has created its own prototype for transforming sketches into workable design code. Using the 150 components within the Airbnb UI system, the prototype’s machine learning algorithm was able to successfully create design code, with just a small amount of intermediary code to enable browser renderings.
The company has also live-coded prototypes from whiteboard drawings, turned production code into design files for front-end/UI designers, and enabled component specifications for engineers from mocks. Airbnb was also able to use their own coded images with a variety of platforms, including React Native, iOS Native, and Android Native. Perhaps most impressive is the fact that the UI stayed the same regardless of what device it was displayed on – a possible solution to the much-loathed browser/device incompatibility issue.
The Future of Front-End Design
A day in the life of a front-end developer largely consisted of turning mockups into usable code. This process can be both tedious and time-consuming. But for most developers, it’s an inescapable reality of the current design process. So, it must be completed before they can move on to more interesting steps. Like building out a site or creating the interesting features that make it unique and user-friendly.
Combining machine learning with front-end design gives developers more time for creative tasks. Instead of spending countless hours trying to decipher sketches or figuring out the graphic design, development and executive teams can simply get down to work. By doing what they do best – design. Best of all, a more streamlined design process will improve how companies in this space operate. Since everyone, right from CEOs to interns, will have a better understanding of the design process. And would be able to quickly try out design ideas with an understanding of protocols. What’s more, time to production would be relatively predictable. Whether you’re a bootstrapped startup or a big enterprise with a custom web portal, there’s one thing you need to consider. The optimized allocation of resources, time management, and design budgets can all add up to serious financial savings and increased ROI.
On a final note
Applying machine learning to front-end design may seem far-fetched. But it’s clear that it has the potential to both revolutionize how companies design. This new process may require front-end developers to become familiar with machine learning and programming languages. Or at least learn to work with different applications. But it’s unlikely to replace the need for them in the near future. This is especially true in industries where the ability to think outside the box and create a truly innovative and personalized user experience is essential to maintaining a competitive advantage. In short, don’t rush to put front-end developers on the endangered species list.
