Our developers managed to run around 750MB per website open once.
They have put in ticket with ops that the server is slow and could we look at it. So we looked. Every single video on a page with long video list pre-loaded a part of it. The single reason the site didn't ran like shit for them is coz office had direct fiber to out datacenter few blocks away.
We really shouldn't allow web developers more than 128kbit of connection speed, anything more and they just make nonsense out of it.
PSA for those who aren’t aware: Chromium/Firefox-based browsers have a Network tab in the developer tools where you can dial down your bandwidth to simulate a slower 3G or 4G connection.
Combined with CPU throttling, it's a decent sanity check to see how well your site will perform on more modest setups.
I once spent around an hour optimizing a feature because it felt slow - turns out that the slower simulated connection had just stayed enabled after a restart (can’t remember if it was just the browser or the OS, but I previously needed it and then later just forgot to turn it off). Good times, useful feature though!
I still test mine on GPRS, because my website should work fine in the Berlin U-Bahn. I also spent a lot of time working from hotels and busses with bad internet, so I care about that stuff.
Developers really ought to test such things better.
CPU/network throttling needs to be set for the product manager and management - that's the only way you might see real change.
We have some egregious slowness in our app that only shows up for our largest customers in production but none of our organizations in development have that much data. I created a load testing organization and keep considering adding management to it so they implicitly get the idea that fixing the slowness is important.
Peanuts! My wife’s workplace has an internal photo gallery page. If your device can cope with it and you wait long enough, it’ll load about 14GB of images (so far). In practice, it will crawl along badly and eventually just crash your browser (or more), especially if you’re on a phone.
The single-line change of adding loading=lazy to the elements wouldn’t fix everything, but it would make the page at least basically usable.
If you want to see context aware pre-fetching done right go to mcmaster.com ...
There are good reasons to have a small cheap development staging server, as the rate-limited connection implicitly trains people what not to include. =3
I used the text web (https://text.npr.org and the like) thru Lyx. Also, Usenet, Gopher, Gemini, some 16 KBPS opus streams, everything under 2.7 KBPS when my phone data plan was throttled and I was using it in tethering mode. Tons of sites did work, but Gopher://magical.fish ran really fast.
Bitlbee saved (and still saves) my ass with tons of the protocols available via IRC using nearly nil data to connect. Also you can connect with any IRC client since early 90's.
Not just web developers. Electron lovers should be trottled with 2GB of RAM machines and some older Celeron/Core Duo machine with a GL 2.1 compatible video card. It it desktop 'app' smooth on that machine, your project it's ready.
You don't even need video for this: I once worked for a company that put a carousel with everything in the product line, and every element was just pointing to the high resolution photography assets: The one that maybe would be useful for full page print media ads. 6000x4000 pngs. It worked fine in the office, they said. Add another nice background that size, a few more to have on the sides as you scroll down...
I was asked to look at the site when it was already live, and some VP of the parent company decided to visit the site from their phone at home.
Should also give designers periodically small displays with low maximum contrast, and have them actually try to achieve everyday tasks with the UX they have designed.
> Our developers managed to run around 750MB per website open once.
This breaks my brain..."run around 750MB" "per website" "open once".
Do you mean your company had several websites (as in unique websites), and each had 750MB? Or the transferred data volume of each was 750MB? Or just the first page? Or the source code?
And do you mean "open once", as in "once upon a time", or if you open the website one time? Or did your developers open and shut websites, and the open ones had 750MB? Or one time when you entered your developer's office space, you saw they had opened several websites on their computers, random ones, and by coincidence you saw the network tabs and each was 750MB data transferred?
this is a general problem with lots of development. Network, Memory, GPU Speed. Designer / Engineer is on a modern Mac with 16-64 gig of ram and fast internet. They never try how their code/design works on some low end Intel UHD 630 or whatever. Lots of developers making 8-13 layer blob backgrounds that runs at 60 for 120fps on their modern mac but at 5-10fps on the average person's PC because of 15x overdraw.
I built an internal dashboard once that displayed thumbnails of building plans. Actually no, I built the dashboard, then the new guy added the thumbnails, but anyway. We didn't actually have a thumbnail generator because the sprint for that was skipped. Most of our users had only a couple of projects so even if each image was several mbs large, it wasn't that huge of an issue for them. The internal dashboard though, loaded a thousand projects per page.
I chewed through 7Gb of data in about 30 minutes while working tethered to my phone.
Well as long as the website was already full loaded and responsive, and the videos show a thumbnail/placeholder, you are not blocked by that. Preloading and even very agressive pre-loading are a thing nowaadays. It is hostile to the user (because it draws their network traffic they pay for) but project managers will often override that to maximize gains from ad revenue.
Just FYI how this generally works: it's not developers who add it, but non-technical people.
Developers only add a single in the page, which loads Google Tag Manager, or similar monstrosity, at the request of someone high up in the company. Initially it loads ~nothing, so it's fine.
Over time, non-technical people slap as many advertising "partner" scripts they can in the config of GTM, straight to prod without telling developers, and without thinking twice about impact on loading times etc. All they track is $ earned on ads.
(It's sneaky because those scripts load async in background so it doesn't immediately feel like the website gets slower / more bloated. And of course, on a high end laptop the website feels "fine" compared to a cheap Android. Also, there's nothing developers can do about those requests, they're under full the control of all those 3rd-parties.)
Fun fact: "performance" in the parlance of adtech people means "ad campaign performance", not "website loading speed". ("What do you mean, performance decreased when we added more tracking?")
Author here. Woke up in today to see this on the front page, thank you to the person who submitted it! Initially, my biggest fear was the HN "Hug of Death" taking it down. Happily, Cloudflare's edge caching absorbed 19.24 GB of bandwidth in a few hours with a 98.5% cache hit ratio, so the origin server barely noticed.
The discussions here about DNS-level blocking and Pi-hole are spot on. It's interesting that the burden of a clean reading experience is slowly being offloaded to the user's network stack.
These days the NYT is in a race to the bottom. I no longer even bother to bypass ads let alone read the news stories because of its page bloat and other annoyances. It's just not worth the effort.
Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
We'll simply cut the headlines from the offending website and past it into a search engine and find another site with the same or similar info but with easier access.
I no longer think about it as by now my actions are automatic. Rarely do I find an important story that's just limited to only one website, generally dozens have the story and because of syndication the alternative site one selects even has identical text and images.
My default browsing is with JavaScript defaulted to "off" and it's rare that I have to enable it (which I can do with just one click).
I never see Ads on my Android phone or PC and that includes YouTube. Disabling JavaScript on webpages nukes just about all ads, they just vanish, any that escape through are then trapped by other means. In ahort, ads are optional. (YouTube doesn't work sans JS, so just use NewPipe or PipePipe to bypass ads.)
Disabling JavaScript also makes pages blindingly fast as all that unnecessary crap isn't loaded. Also, sans JS it's much harder for websites to violate one's privacy and sell one's data.
Do I feel guilty about skimming off info in this manner? No, not the slightest bit. If these sites played fair then it'd be a different matter but they don't. As they act like sleazebags they deserve to be treated as such.
My family's first broadband internet connection, circa 2005, came with a monthly data quota of 400 MB.
The fundamental problem of journalism is that the economics no longer works out. Historically, the price of a copy of a newspaper barely covered the cost of printing; the rest of the cost was covered by advertising. And there was an awful lot of advertising: everything was advertised in newspapers. Facebook Marketplace and Craigslist were a section of the newspaper, as was whichever website you check for used cars or real estate listings. Journalism had to be subsidised by advertising, because most people aren't actually that interested in the news to pay the full cost of quality reporting; nowadays, the only newspapers that are thriving are those that aggressively target those who have an immediate financial interest in knowing what's going on: the Financial Times, Bloomberg, and so on.
The fact is that for most people, the news was interesting because it was new every day. Now that there is a more compelling flood of entertainment in television and the internet, news reporting is becoming a niche product.
The lengths that news websites are going to to extract data from their readers to sell to data brokers is just a last-ditch attempt to remain profitable.
I just loaded the nytimes.com page as an experiment. The volume of tracking pixels and other ad non-sense is truly horrifying.
But at least in terms of the headline metric of bandwidth, it's somewhat less horrifying. With my ad-blocker off, Firefox showed 44.47mb transferred. Of that 36.30mb was mp4 videos. These videos were journalistic in nature (they were not ads).
So, yes in general, this is like the Hindenburg of web pages. But I still think it's worth noting that 80% of that headline bandwidth is videos, which is just part of the site's content. One could argue that it is too video heavy, but that's an editorial issue, not an engineering issue.
Allowing scripting on websites (in the mid-90s) was a completely wrong decision. And an outrage. Programs are downloaded to my computer and executed without me being able to review them first—or rely on audits by people I trust. That’s completely unacceptable; it’s fundamentally flawed.
Of course, you disable scripts on websites. But there are sites that are so broken that they no longer work properly, since the developers are apparently so confused that they assume people only view their pages with JavaScript enabled.
It would have been so much better if we had simply decided back in the ’90s that executable programs and HTML don’t belong together. The world would be so much better today.
375 comments
They have put in ticket with ops that the server is slow and could we look at it. So we looked. Every single video on a page with long video list pre-loaded a part of it. The single reason the site didn't ran like shit for them is coz office had direct fiber to out datacenter few blocks away.
We really shouldn't allow web developers more than 128kbit of connection speed, anything more and they just make nonsense out of it.
Combined with CPU throttling, it's a decent sanity check to see how well your site will perform on more modest setups.
Developers really ought to test such things better.
We have some egregious slowness in our app that only shows up for our largest customers in production but none of our organizations in development have that much data. I created a load testing organization and keep considering adding management to it so they implicitly get the idea that fixing the slowness is important.
The single-line change of adding loading=lazy to the
elements wouldn’t fix everything, but it would make the page at least basically usable.
> We really shouldn't allow web developers more than 128kbit
Marketing dept. too. They're the primary culprits in all the tracking scripts.
There are good reasons to have a small cheap development staging server, as the rate-limited connection implicitly trains people what not to include. =3
Bitlbee saved (and still saves) my ass with tons of the protocols available via IRC using nearly nil data to connect. Also you can connect with any IRC client since early 90's.
Not just web developers. Electron lovers should be trottled with 2GB of RAM machines and some older Celeron/Core Duo machine with a GL 2.1 compatible video card. It it desktop 'app' smooth on that machine, your project it's ready.
I was asked to look at the site when it was already live, and some VP of the parent company decided to visit the site from their phone at home.
> Our developers managed to run around 750MB per website open once.
This breaks my brain..."run around 750MB" "per website" "open once".
Do you mean your company had several websites (as in unique websites), and each had 750MB? Or the transferred data volume of each was 750MB? Or just the first page? Or the source code?
And do you mean "open once", as in "once upon a time", or if you open the website one time? Or did your developers open and shut websites, and the open ones had 750MB? Or one time when you entered your developer's office space, you saw they had opened several websites on their computers, random ones, and by coincidence you saw the network tabs and each was 750MB data transferred?
I wouldn't even guarantee it's developers adding it. I'm sure they have some sort of content management system for doing article and ad layout.
I chewed through 7Gb of data in about 30 minutes while working tethered to my phone.
> 422 network requests and 49 megabytes of data
Just FYI how this generally works: it's not developers who add it, but non-technical people.
Developers only add a single
in the page, which loads Google Tag Manager, or similar monstrosity, at the request of someone high up in the company. Initially it loads ~nothing, so it's fine.Over time, non-technical people slap as many advertising "partner" scripts they can in the config of GTM, straight to prod without telling developers, and without thinking twice about impact on loading times etc. All they track is $ earned on ads.
(It's sneaky because those scripts load async in background so it doesn't immediately feel like the website gets slower / more bloated. And of course, on a high end laptop the website feels "fine" compared to a cheap Android. Also, there's nothing developers can do about those requests, they're under full the control of all those 3rd-parties.)
Fun fact: "performance" in the parlance of adtech people means "ad campaign performance", not "website loading speed". ("What do you mean, performance decreased when we added more tracking?")
The discussions here about DNS-level blocking and Pi-hole are spot on. It's interesting that the burden of a clean reading experience is slowly being offloaded to the user's network stack.
Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
We'll simply cut the headlines from the offending website and past it into a search engine and find another site with the same or similar info but with easier access.
I no longer think about it as by now my actions are automatic. Rarely do I find an important story that's just limited to only one website, generally dozens have the story and because of syndication the alternative site one selects even has identical text and images.
My default browsing is with JavaScript defaulted to "off" and it's rare that I have to enable it (which I can do with just one click).
I never see Ads on my Android phone or PC and that includes YouTube. Disabling JavaScript on webpages nukes just about all ads, they just vanish, any that escape through are then trapped by other means. In ahort, ads are optional. (YouTube doesn't work sans JS, so just use NewPipe or PipePipe to bypass ads.)
Disabling JavaScript also makes pages blindingly fast as all that unnecessary crap isn't loaded. Also, sans JS it's much harder for websites to violate one's privacy and sell one's data.
Do I feel guilty about skimming off info in this manner? No, not the slightest bit. If these sites played fair then it'd be a different matter but they don't. As they act like sleazebags they deserve to be treated as such.
The fundamental problem of journalism is that the economics no longer works out. Historically, the price of a copy of a newspaper barely covered the cost of printing; the rest of the cost was covered by advertising. And there was an awful lot of advertising: everything was advertised in newspapers. Facebook Marketplace and Craigslist were a section of the newspaper, as was whichever website you check for used cars or real estate listings. Journalism had to be subsidised by advertising, because most people aren't actually that interested in the news to pay the full cost of quality reporting; nowadays, the only newspapers that are thriving are those that aggressively target those who have an immediate financial interest in knowing what's going on: the Financial Times, Bloomberg, and so on.
The fact is that for most people, the news was interesting because it was new every day. Now that there is a more compelling flood of entertainment in television and the internet, news reporting is becoming a niche product.
The lengths that news websites are going to to extract data from their readers to sell to data brokers is just a last-ditch attempt to remain profitable.
But at least in terms of the headline metric of bandwidth, it's somewhat less horrifying. With my ad-blocker off, Firefox showed 44.47mb transferred. Of that 36.30mb was mp4 videos. These videos were journalistic in nature (they were not ads).
So, yes in general, this is like the Hindenburg of web pages. But I still think it's worth noting that 80% of that headline bandwidth is videos, which is just part of the site's content. One could argue that it is too video heavy, but that's an editorial issue, not an engineering issue.
It would have been so much better if we had simply decided back in the ’90s that executable programs and HTML don’t belong together. The world would be so much better today.