Skip to main content
Slight Future web development — server configuration, performance, bot management, and front-end notes

Web Development

The web development section covers the operational reality of building and maintaining web infrastructure — the server configuration decisions, the compression tuning, the bot traffic management, the front-end behaviour quirks, and the content protection strategies that determine whether a site works well in production or just passes a synthetic audit. Each page focuses on a specific problem encountered during real web operations, documented with enough technical detail to be directly useful.

The scope spans Apache server configuration, HTTP compression with Brotli and gzip, crawler and bot behaviour analysis, client-side HTML5 quirks, content scraping defence, and web application icon handling across platforms. These are not tutorial-style introductions to web development concepts. They are the notes that result from running production infrastructure, observing unexpected behaviour, and documenting the working solution alongside the context that explains why it works.

This section connects naturally with the tech notes on protocol behaviour and platform internals, and with the security section where server hardening meets web infrastructure. The topics index offers cross-section navigation for themes like web performance that span multiple editorial sections.

Server configuration and performance​

The performance of a web server depends on decisions that interact in non-obvious ways. Compression settings affect cache efficiency. Cache headers affect CDN behaviour. Bot traffic affects the capacity available for real users. These pages address the specific configuration challenges that arise in production.

Apache mod_brotli​

Brotli compression offers measurably better ratios than gzip for text-based content, but configuring mod_brotli on Apache involves more nuance than enabling the module and setting a compression level. This page covers the module configuration in detail — the compression level trade-offs (level 11 is excellent for pre-compressed static assets and disastrous for dynamic responses), the interaction with proxy caching and the Vary header, the content types worth compressing, and the fallback behaviour when a client does not support Brotli.

The page also addresses the operational question of when Brotli actually matters versus when gzip is sufficient. For sites served through a CDN that handles compression at the edge, the server-side configuration decisions are different from sites served directly. If you are configuring Apache compression and want to understand the trade-offs rather than just copy a configuration block, this is the reference.

Hotlink protection — preventing other sites from embedding your images and consuming your bandwidth — sounds like a solved problem from 2005, but the modern web has reintroduced the challenge in new forms. CDN behaviour, the Referer header's diminished reliability, privacy-focused browser changes that strip or modify referrer information, and the rise of server-side rendering that fetches resources without a browser referrer all mean that traditional Referer-based hotlink protection no longer works the way it once did.

This page documents the current state of hotlink protection: what still works, what does not, and the approaches that provide meaningful protection without breaking legitimate use cases like search engine image previews, social media cards, and RSS reader image display. It is written for site operators who are seeing unexpected bandwidth usage from embedded resources and want a solution that accounts for how the modern web actually sends (or does not send) referrer information.

Bot and crawler management​

Bots consume real server resources, and the line between a helpful search engine crawler and an abusive scraper is not always where you expect it to be. These pages address the practical challenges of managing automated traffic.

Excessive AppleNewsBot Requests​

AppleNewsBot can generate surprising volumes of requests to a site, particularly when content is formatted for Apple News and the bot is prefetching aggressively. This page documents the observed request patterns — the volume, the frequency, the user agent variations, and the server-side impact. It covers the practical options for managing the traffic: robots.txt directives, rate limiting, and the trade-offs involved in restricting a bot that also drives traffic to your content through Apple News.

The investigation is useful for any site operator who has noticed AppleNewsBot in their access logs and is trying to determine whether the request volume is normal, whether it is affecting server performance, and what the practical options are for controlling it without losing Apple News distribution.

Defacing Content Scrapers​

Content scrapers steal your work, and the traditional responses — DMCA takedowns, Referer checks, IP blocking — are slow, incomplete, or easily circumvented. This page documents an alternative approach: identifying scraper traffic through its request patterns and serving modified content that disrupts the scraped copy. The technique is not foolproof, but it imposes a cost on automated theft that passive protection does not.

The page covers the detection heuristics, the server-side implementation, the ethical considerations (you are deliberately serving different content to different clients, which has implications), and the practical effectiveness observed over time. If you have dealt with content theft and found the conventional remedies inadequate, this page offers a more active defensive strategy.

Front-end behaviour​

Client-side web development involves working within a platform where specification, implementation, and user expectation do not always align. These pages document specific front-end behaviours that cause real problems.

HTML5 Input Number Localization​

The HTML5 <input type="number"> element handles locale-dependent number formatting in ways that break form submissions for users in countries that use a comma as the decimal separator. This page documents the specific browser behaviours — which browsers accept comma input, which silently reject it, which convert it, and what the server receives in each case. The problem is well known in European web development circles but poorly documented in a way that covers all the browsers and the server-side implications simultaneously.

If you have received bug reports about number input fields silently submitting empty values or incorrect numbers, and the issue only affects users in certain locales, this page explains exactly what is happening and the practical solutions — from inputmode attributes to server-side parsing adjustments — that address it.

GNOME Web App Icons​

Installing a web application as a standalone app from GNOME Web (Epiphany) involves icon handling that differs from the web app manifest specification in ways that affect the installed app's appearance. This page documents how GNOME Web selects, downloads, and displays web app icons, where its behaviour diverges from the specification and from Chromium-based browsers, and the markup adjustments that produce correct icon display across platforms.

The investigation is relevant for any developer shipping a progressive web app and wanting it to install correctly on Linux desktops running GNOME, not just on Chrome OS and Android where the testing usually happens.

What readers usually need​

Web development questions in this section tend to fall into a few categories:

For protocol-level details that underpin these topics, the tech notes section covers DNS, networking, and platform internals. For security-specific aspects of web infrastructure, the security section addresses TLS, server hardening, and vulnerability analysis. The topics index provides cross-section views of themes like web performance and the open web.