Every self-respecting superpower has its special services. Who among us has not at least heard of MI6 and its most famous agent, James Bond? Brilliant, effective, flawless, deadly handsome, and a gentleman on top of that. Ian Fleming created a great character. So why should we not have our own special agent in the world of web development?

Special Agent

Well, my dear readers, we do have one. Maybe he is not brilliant, effective, or deadly handsome. Maybe he is not the best at winning hearts either, but... he exists. And this is probably where his list of advantages ends. His name is Agent, User Agent.

Let us start with a technical look at the topic. By definition, a User Agent is software that, on behalf of the user, fetches, renders, and helps use resources from websites. In other words, its main task is to support the user in easier and more pleasant surfing through the web. Every piece of software implemented with web technologies has this agent. Where do we meet it? Above all in web browsers, media players, and all kinds of plugins that use internet content.

Since every piece of software based on web technologies is equipped with this agent, how can we use it? People started asking this question already in the first half of the 1990s.

Something in the HTTP Header

User Agent, as an HTTP header, was described in RFC 1945 from May 1996. In practice, it was defined there as a special HTTP header responsible for sending information about the user agent that initiates the request. The goal behind it was mainly connected with security and statistics. By introducing it, people wanted to get more information about HTTP protocol violations, and also, through proper identification of the requester, adjust the response more effectively. Such adjustment can be quite valuable when the requester has certain limitations in handling a response. De facto, it gave a chance to fit the technology better to the capabilities of the recipient.

The sent header contains special product tokens, which are simply identifiers of the applications sending the request. They include names or code names, an optional slash, and versions. Apart from tokens, the header can also contain comments meant to help with identification, as well as any dependent products that form an important part of the sending application.

The basic structure of the header looks like this:

`User-Agent: <product> / <product-version> <comment>`

A common template for web technologies looks like this:

`User-Agent: Mozilla/5.0 (<system-information>) <platform> (<platform-details>) <extensions>`

Notice that, apart from the basic structure, we also find information about the operating system, engine version, and extensions added to the User Agent. Quite a lot of information, right?

Just Hello

The original idea behind sending the User-Agent header was connected with identification. In practice, it came down to a simple greeting, for example: "Hi, I am Mozilla Firefox, I am running on Windows." The knowledge we get from such a header gives us quite a few possibilities. We can inform the user that they are using an old browser version, or even force an update. We can apply different styling and scripting depending on the browser. We can limit certain features depending on the browser version and engine, or even the operating system. The problem is that User-Agent is not friendly, and at first glance it is not simply "hello". Let us analyze a few headers:

`Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0`

The example above points to Mozilla Firefox version 12, running on the Windows 7 operating system in a 64-bit version. Windows 7 has the code name Windows NT 6.1, and WOW64 means a 64-bit version of the system. Good, so let us look at the User Agent for Internet Explorer:

`Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)`

MSIE 9.0 means we are dealing with Internet Explorer version 9. Trident/5.0 is the name and version of the rendering engine. The whole thing is running on Windows 7 in a 64-bit version. Wait a second... IE introduced itself as Mozilla.

It gets even more interesting with Chrome.

`Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5`

Here we can see not only Mozilla and Gecko, the Firefox rendering engine, but also Safari. I think we need to look at this more closely.

The First Browsers

When people tell the history of browsers, they often start with Netscape Navigator. Not without reason. Netscape was a precursor of many technologies important to the internet world, such as frames, JavaScript, and DHTML. It also had a fairly impressive number of users. By mid-1995, more than 80% of internet users were using it. When the end of support and the end of further development of the browser were announced at the end of 2007, it was also stated that the remains of the Netscape Navigator family would be transferred to the Mozilla Foundation.

Why Mozilla specifically? Mozilla was the code name of the Netscape Navigator browser. When, in the early 2000s, AOL laid off all programmers from Netscape Communications Corporation working on that browser, it became necessary to create an entity that could continue developing and maintaining the project. That is how the Mozilla Foundation was brought to life, and its main task was to take care of the Mozilla project, meaning primarily the browser mentioned above.

Netscape, however, was not the first browser on the market. It was built on top of a browser called Mosaic. Its User Agent was `NCSA_Mosaic/2.0`. Mosaic would probably have existed longer if not for the fact that the Mozilla browser was created, later renamed Netscape. The User Agent for the new browser was `Mozilla/1.0`. OK, so now we know what Mozilla means, but why did it become so popular and so omnipresent?

For the curious, I will add that Mozilla was also the name of the green lizard that served as Netscape's mascot.

The mascot was created in 1994. That year definitely belonged to Jurassic Park. No one from the creative team wanted a dinosaur as a mascot, but they did not have much choice. That is how Mozilla was born, originally called the "Mosaic killer".

As I mentioned earlier, Netscape introduced support for HTML frames. And that was a breakthrough...

Frames - the Beginning of Chaos

Frames turned out to be a game changer. Before they were introduced, websites were built only in plain HTML, without a server and without HTML preprocessors. Changes to shared elements, such as headers or footers, had to be made in every file where those elements appeared. For websites with many subpages, this was very tiring and required a lot of work. The introduction of frames made the process much easier, because they allowed different HTML documents to be nested inside one another. Frames became so popular that they were officially introduced into the HTML specification in 1998. You understand that correctly: Netscape introduced frames before they were described in the HTML language specification. Quite a move.

Servers started checking whether the User Agent contained the word Mozilla and, based on that, decided which page to send to the user: one built with frames, so the more modern version, or one without them. The market was developing, and there were more and more players. The next browser that introduced support for frames was Internet Explorer. Unfortunately, servers did not send the more modern page to IE, because its User Agent did not contain the magic word Mozilla.

What did Microsoft do in that situation? The easiest thing. It changed its User Agent and added the word Mozilla. In the comments, it also added information about compatibility and IE itself. The next browsers followed exactly the same path.

As a rule, servers returned the more modern version of a page to Netscape, and later to Firefox. So they looked for words in User Agents connected with those tools, such as Gecko. Gecko is Firefox's rendering engine. Other engine creators began adding their own labels to comments that were, in practice, meant to tell the server that they wanted the more modern version of the page. That is how the phrase `like Gecko` started being added to the description of KHTML, the engine for the Konqueror browser originally available on Linux. WebKit, the rendering engine used by Chrome, was then created on top of KHTML. WebKit kept the original KHTML label. So Chrome has the words `Mozilla`, `KHTML like Gecko`, and `Safari` in its User Agent, even though they have very little to do with its identity.

The conclusion? For the server, it does not really matter what is inside the User Agent. The server looks for specific words and, based on them, decides what to send to the user. In one word: chaos.

The frame mechanism is still available in every modern browser, even though already in XHTML 1.1, in 2001, it was considered an obsolete technology.

A Matter of Trust

So how do we check whether a given browser supports a given feature? Our special agent is already so cluttered that we may have certain problems trusting it. Another solution may be to use libraries that check whether particular features are available. One of them is Modernizr, which tests the availability of different features, CSS and JS, inside the browser, then creates a global object containing information about their availability for JavaScript and adds appropriate CSS classes to the document's body element. Useful, especially because it does not rely on our User Agent.

There is one more issue connected with trust. Google decided to treat User Agent as obsolete. In January 2020, Yoav Weiss, a Google engineer, wrote:

Well then. Since everyone lies and pretends to be other browsers, maybe it really is worth thinking about another identification mechanism. Unfortunately, as in the quote above, Safari failed, and Chrome has still not fully dealt with it since 2020.

Summary

This special agent has made quite a mess. There are conflicts, explosions, lies, and romances. User Agent, in the form I presented above, still exists, and we are in 2022. It was supposed to be something that would help identify individual players on the market, provide access to the most modern technologies, and take care of security. Because of delays from creators and taking shortcuts, it became something we have to live with and cannot get rid of, because we do not really have a good idea how, or with whom, to replace it.

Although a new agent is already on the way... Hints, Client Hints. I wonder whether it will be more effective.