Our developers managed to run around 750MB per website open once.
They have put in ticket with ops that the server is slow and could we look at it. So we looked. Every single video on a page with long video list pre-loaded a part of it. The single reason the site didn't ran like shit for them is coz office had direct fiber to out datacenter few blocks away.
We really shouldn't allow web developers more than 128kbit of connection speed, anything more and they just make nonsense out of it.
PSA for those who aren’t aware: Chromium/Firefox-based browsers have a Network tab in the developer tools where you can dial down your bandwidth to simulate a slower 3G or 4G connection.
Combined with CPU throttling, it's a decent sanity check to see how well your site will perform on more modest setups.
Peanuts! My wife’s workplace has an internal photo gallery page. If your device can cope with it and you wait long enough, it’ll load about 14GB of images (so far). In practice, it will crawl along badly and eventually just crash your browser (or more), especially if you’re on a phone.
The single-line change of adding loading=lazy to the elements wouldn’t fix everything, but it would make the page at least basically usable.
If you want to see context aware pre-fetching done right go to mcmaster.com ...
There are good reasons to have a small cheap development staging server, as the rate-limited connection implicitly trains people what not to include. =3
I used the text web (https://text.npr.org and the like) thru Lyx. Also, Usenet, Gopher, Gemini, some 16 KBPS opus streams, everything under 2.7 KBPS when my phone data plan was throttled and I was using it in tethering mode. Tons of sites did work, but Gopher://magical.fish ran really fast.
Bitlbee saved (and still saves) my ass with tons of the protocols available via IRC using nearly nil data to connect. Also you can connect with any IRC client since early 90's.
Not just web developers. Electron lovers should be trottled with 2GB of RAM machines and some older Celeron/Core Duo machine with a GL 2.1 compatible video card. It it desktop 'app' smooth on that machine, your project it's ready.
You don't even need video for this: I once worked for a company that put a carousel with everything in the product line, and every element was just pointing to the high resolution photography assets: The one that maybe would be useful for full page print media ads. 6000x4000 pngs. It worked fine in the office, they said. Add another nice background that size, a few more to have on the sides as you scroll down...
I was asked to look at the site when it was already live, and some VP of the parent company decided to visit the site from their phone at home.
Should also give designers periodically small displays with low maximum contrast, and have them actually try to achieve everyday tasks with the UX they have designed.
> Our developers managed to run around 750MB per website open once.
This breaks my brain..."run around 750MB" "per website" "open once".
Do you mean your company had several websites (as in unique websites), and each had 750MB? Or the transferred data volume of each was 750MB? Or just the first page? Or the source code?
And do you mean "open once", as in "once upon a time", or if you open the website one time? Or did your developers open and shut websites, and the open ones had 750MB? Or one time when you entered your developer's office space, you saw they had opened several websites on their computers, random ones, and by coincidence you saw the network tabs and each was 750MB data transferred?
Just FYI how this generally works: it's not developers who add it, but non-technical people.
Developers only add a single in the page, which loads Google Tag Manager, or similar monstrosity, at the request of someone high up in the company. Initially it loads ~nothing, so it's fine.
Over time, non-technical people slap as many advertising "partner" scripts they can in the config of GTM, straight to prod without telling developers, and without thinking twice about impact on loading times etc. All they track is $ earned on ads.
(It's sneaky because those scripts load async in background so it doesn't immediately feel like the website gets slower / more bloated. And of course, on a high end laptop the website feels "fine" compared to a cheap Android. Also, there's nothing developers can do about those requests, they're under full the control of all those 3rd-parties.)
Fun fact: "performance" in the parlance of adtech people means "ad campaign performance", not "website loading speed". ("What do you mean, performance decreased when we added more tracking?")
Author here. Woke up in today to see this on the front page, thank you to the person who submitted it! Initially, my biggest fear was the HN "Hug of Death" taking it down. Happily, Cloudflare's edge caching absorbed 19.24 GB of bandwidth in a few hours with a 98.5% cache hit ratio, so the origin server barely noticed.
The discussions here about DNS-level blocking and Pi-hole are spot on. It's interesting that the burden of a clean reading experience is slowly being offloaded to the user's network stack.
These days the NYT is in a race to the bottom. I no longer even bother to bypass ads let alone read the news stories because of its page bloat and other annoyances. It's just not worth the effort.
Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
We'll simply cut the headlines from the offending website and past it into a search engine and find another site with the same or similar info but with easier access.
I no longer think about it as by now my actions are automatic. Rarely do I find an important story that's just limited to only one website, generally dozens have the story and because of syndication the alternative site one selects even has identical text and images.
My default browsing is with JavaScript defaulted to "off" and it's rare that I have to enable it (which I can do with just one click).
I never see Ads on my Android phone or PC and that includes YouTube. Disabling JavaScript on webpages nukes just about all ads, they just vanish, any that escape through are then trapped by other means. In ahort, ads are optional. (YouTube doesn't work sans JS, so just use NewPipe or PipePipe to bypass ads.)
Disabling JavaScript also makes pages blindingly fast as all that unnecessary crap isn't loaded. Also, sans JS it's much harder for websites to violate one's privacy and sell one's data.
Do I feel guilty about skimming off info in this manner? No, not the slightest bit. If these sites played fair then it'd be a different matter but they don't. As they act like sleazebags they deserve to be treated as such.
My family's first broadband internet connection, circa 2005, came with a monthly data quota of 400 MB.
The fundamental problem of journalism is that the economics no longer works out. Historically, the price of a copy of a newspaper barely covered the cost of printing; the rest of the cost was covered by advertising. And there was an awful lot of advertising: everything was advertised in newspapers. Facebook Marketplace and Craigslist were a section of the newspaper, as was whichever website you check for used cars or real estate listings. Journalism had to be subsidised by advertising, because most people aren't actually that interested in the news to pay the full cost of quality reporting; nowadays, the only newspapers that are thriving are those that aggressively target those who have an immediate financial interest in knowing what's going on: the Financial Times, Bloomberg, and so on.
The fact is that for most people, the news was interesting because it was new every day. Now that there is a more compelling flood of entertainment in television and the internet, news reporting is becoming a niche product.
The lengths that news websites are going to to extract data from their readers to sell to data brokers is just a last-ditch attempt to remain profitable.
I just loaded the nytimes.com page as an experiment. The volume of tracking pixels and other ad non-sense is truly horrifying.
But at least in terms of the headline metric of bandwidth, it's somewhat less horrifying. With my ad-blocker off, Firefox showed 44.47mb transferred. Of that 36.30mb was mp4 videos. These videos were journalistic in nature (they were not ads).
So, yes in general, this is like the Hindenburg of web pages. But I still think it's worth noting that 80% of that headline bandwidth is videos, which is just part of the site's content. One could argue that it is too video heavy, but that's an editorial issue, not an engineering issue.
Allowing scripting on websites (in the mid-90s) was a completely wrong decision. And an outrage. Programs are downloaded to my computer and executed without me being able to review them first—or rely on audits by people I trust. That’s completely unacceptable; it’s fundamentally flawed.
Of course, you disable scripts on websites. But there are sites that are so broken that they no longer work properly, since the developers are apparently so confused that they assume people only view their pages with JavaScript enabled.
It would have been so much better if we had simply decided back in the ’90s that executable programs and HTML don’t belong together. The world would be so much better today.
I also use and like the comparison in units of Windows 95 installs (~40MB), which is also rather ironic in that Win95 was widely considered bloated when it was released.
While this article focuses on ads, it's worth noting that sites have had ads for a long time, but it's their obnoxiousness and resource usage that's increased wildly over time. I wouldn't mind small sponsored links and (non-animated!) banners, but the moment I enable JS to read an article and it results in a flurry of shit flying all over the page and trying to get my attention, I leave promptly.
Not only are loading times and total network usage ridiculous, sites will continue to violate your privacy via trackers and waste your CPU even when background idling. I've written about these issues a few times in the last few years, so just sharing for those interested:
This is just the top of the iceberg. Don't get me started on airlines websites (looking at you Air Canada), where the product owner, designers, developers are not able to get a simple workflow straight without loading Mb of useless javascript and interrupt the user journey multiple times.
Give me back the command line terminal like Amadeus, that would be perfect.
How can we go back to a Web where websites are designed to be used by the user and not for the shareholders?
Modern web dev is ridiculous. Most websites are an ad ridden tracking hellacape. Seeing sites like hn where lines of js are taken seriously is a godsend. Make the web less bloated.
This is why people continue to lament Google Reader (and RSS in general): it was a way to read content on your own terms, without getting hijacked by ads.
I remember in 2008, when Wizards of the Coast re-launched the official Dungeons & Dragons website to coincide with the announcement of the fourth edition rules. The site was something in the region of 4 MB, plus a 20 MB embedded video file. A huge number of people were refreshing the site to see what the announcement was, and it was completely slammed. Nobody could watch the trailer until they uploaded it to YouTube later.
4 MB was an absurd size for a website in 2008. It's still an absurd size for a website.
This rubbish also exists disproportionately for recipe pages/cooking websites as well.
You have 20 ads scattered around, an autoplaying video of some random recipe/ad, 2-3 popups to subscribe, buy some affiliated product and then the author's life story and then a story ABOUT the recipe before I am able to see the detailed recipe in the proper format.
It's second nature to open all these websites in reader mode for me atp.
It's really hard to consider any kind of web dev as "engineering." Outcomes like this show that they don't have any particular care for constraints. It's throw-spaghetti-at-the-wall YOLO programming.
It's almost criminal that the article does not mention network-wide DNS blocklists as an obvious solution to this problem. I stop nearly 100% of ads in their tracks using the Hagezi ultimate list, and run uBlock on desktop for cosmetic filtering and YouTube.
I should really run some to tests to figure out how much lighter the load on my link is thanks to the filter.
I also manually added some additional domains (mostly fonts by Google and Adobe) to further reduce load and improve privacy.
> users are greeted by what I call Z-Index Warfare
Nice term!
> Or better yet, inject the newsletter signup as a styled, non-intrusive div between paragraphs 4 and 5. If the user has scrolled that far, they are engaged.
They're engaged with the content! There is no way to make some irrelevant signup "non-intrusive". It's similar to links to unrelated articles - do you want users to actually read the article or jump around reading headlines?
375 comments
They have put in ticket with ops that the server is slow and could we look at it. So we looked. Every single video on a page with long video list pre-loaded a part of it. The single reason the site didn't ran like shit for them is coz office had direct fiber to out datacenter few blocks away.
We really shouldn't allow web developers more than 128kbit of connection speed, anything more and they just make nonsense out of it.
Combined with CPU throttling, it's a decent sanity check to see how well your site will perform on more modest setups.
The single-line change of adding loading=lazy to the
elements wouldn’t fix everything, but it would make the page at least basically usable.
> We really shouldn't allow web developers more than 128kbit
Marketing dept. too. They're the primary culprits in all the tracking scripts.
There are good reasons to have a small cheap development staging server, as the rate-limited connection implicitly trains people what not to include. =3
Bitlbee saved (and still saves) my ass with tons of the protocols available via IRC using nearly nil data to connect. Also you can connect with any IRC client since early 90's.
Not just web developers. Electron lovers should be trottled with 2GB of RAM machines and some older Celeron/Core Duo machine with a GL 2.1 compatible video card. It it desktop 'app' smooth on that machine, your project it's ready.
I was asked to look at the site when it was already live, and some VP of the parent company decided to visit the site from their phone at home.
> Our developers managed to run around 750MB per website open once.
This breaks my brain..."run around 750MB" "per website" "open once".
Do you mean your company had several websites (as in unique websites), and each had 750MB? Or the transferred data volume of each was 750MB? Or just the first page? Or the source code?
And do you mean "open once", as in "once upon a time", or if you open the website one time? Or did your developers open and shut websites, and the open ones had 750MB? Or one time when you entered your developer's office space, you saw they had opened several websites on their computers, random ones, and by coincidence you saw the network tabs and each was 750MB data transferred?
> 422 network requests and 49 megabytes of data
Just FYI how this generally works: it's not developers who add it, but non-technical people.
Developers only add a single
in the page, which loads Google Tag Manager, or similar monstrosity, at the request of someone high up in the company. Initially it loads ~nothing, so it's fine.Over time, non-technical people slap as many advertising "partner" scripts they can in the config of GTM, straight to prod without telling developers, and without thinking twice about impact on loading times etc. All they track is $ earned on ads.
(It's sneaky because those scripts load async in background so it doesn't immediately feel like the website gets slower / more bloated. And of course, on a high end laptop the website feels "fine" compared to a cheap Android. Also, there's nothing developers can do about those requests, they're under full the control of all those 3rd-parties.)
Fun fact: "performance" in the parlance of adtech people means "ad campaign performance", not "website loading speed". ("What do you mean, performance decreased when we added more tracking?")
The discussions here about DNS-level blocking and Pi-hole are spot on. It's interesting that the burden of a clean reading experience is slowly being offloaded to the user's network stack.
Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
We'll simply cut the headlines from the offending website and past it into a search engine and find another site with the same or similar info but with easier access.
I no longer think about it as by now my actions are automatic. Rarely do I find an important story that's just limited to only one website, generally dozens have the story and because of syndication the alternative site one selects even has identical text and images.
My default browsing is with JavaScript defaulted to "off" and it's rare that I have to enable it (which I can do with just one click).
I never see Ads on my Android phone or PC and that includes YouTube. Disabling JavaScript on webpages nukes just about all ads, they just vanish, any that escape through are then trapped by other means. In ahort, ads are optional. (YouTube doesn't work sans JS, so just use NewPipe or PipePipe to bypass ads.)
Disabling JavaScript also makes pages blindingly fast as all that unnecessary crap isn't loaded. Also, sans JS it's much harder for websites to violate one's privacy and sell one's data.
Do I feel guilty about skimming off info in this manner? No, not the slightest bit. If these sites played fair then it'd be a different matter but they don't. As they act like sleazebags they deserve to be treated as such.
The fundamental problem of journalism is that the economics no longer works out. Historically, the price of a copy of a newspaper barely covered the cost of printing; the rest of the cost was covered by advertising. And there was an awful lot of advertising: everything was advertised in newspapers. Facebook Marketplace and Craigslist were a section of the newspaper, as was whichever website you check for used cars or real estate listings. Journalism had to be subsidised by advertising, because most people aren't actually that interested in the news to pay the full cost of quality reporting; nowadays, the only newspapers that are thriving are those that aggressively target those who have an immediate financial interest in knowing what's going on: the Financial Times, Bloomberg, and so on.
The fact is that for most people, the news was interesting because it was new every day. Now that there is a more compelling flood of entertainment in television and the internet, news reporting is becoming a niche product.
The lengths that news websites are going to to extract data from their readers to sell to data brokers is just a last-ditch attempt to remain profitable.
But at least in terms of the headline metric of bandwidth, it's somewhat less horrifying. With my ad-blocker off, Firefox showed 44.47mb transferred. Of that 36.30mb was mp4 videos. These videos were journalistic in nature (they were not ads).
So, yes in general, this is like the Hindenburg of web pages. But I still think it's worth noting that 80% of that headline bandwidth is videos, which is just part of the site's content. One could argue that it is too video heavy, but that's an editorial issue, not an engineering issue.
It would have been so much better if we had simply decided back in the ’90s that executable programs and HTML don’t belong together. The world would be so much better today.
While this article focuses on ads, it's worth noting that sites have had ads for a long time, but it's their obnoxiousness and resource usage that's increased wildly over time. I wouldn't mind small sponsored links and (non-animated!) banners, but the moment I enable JS to read an article and it results in a flurry of shit flying all over the page and trying to get my attention, I leave promptly.
A comparison of CPU usage for idling popular webpages: https://ericra.com/writing/site_cpu.html
Regarding tracker domains on the New Yorker site: https://ericra.com/writing/tracker_new_yorker.html
How can we go back to a Web where websites are designed to be used by the user and not for the shareholders?
https://idlewords.com/talks/website_obesity.htm
4 MB was an absurd size for a website in 2008. It's still an absurd size for a website.
You have 20 ads scattered around, an autoplaying video of some random recipe/ad, 2-3 popups to subscribe, buy some affiliated product and then the author's life story and then a story ABOUT the recipe before I am able to see the detailed recipe in the proper format.
It's second nature to open all these websites in reader mode for me atp.
Running this on the dev console makes it snappy again:
I should really run some to tests to figure out how much lighter the load on my link is thanks to the filter.
I also manually added some additional domains (mostly fonts by Google and Adobe) to further reduce load and improve privacy.
> users are greeted by what I call Z-Index Warfare
Nice term!
> Or better yet, inject the newsletter signup as a styled, non-intrusive div between paragraphs 4 and 5. If the user has scrolled that far, they are engaged.
They're engaged with the content! There is no way to make some irrelevant signup "non-intrusive". It's similar to links to unrelated articles - do you want users to actually read the article or jump around reading headlines?