Improving storage efficiency in Magic Pocket, Dropbox's immutable blob store (dropbox.tech)

by laluser 21 comments 60 points
Read article View on HN

21 comments

[−] nopurpose 36d ago

> Last year, we rolled out a new service that changed how data is placed across Magic Pocket. The change reduced write amplification for background writes, so each write triggered fewer backend storage operations. But it also had an unintended side effect: fragmentation increased, pushing storage overhead higher. Most of that growth came from a small number of severely under-filled volumes that consumed a disproportionate share of raw capacity

Me thinking big corps with huge infrastructure bills meticulously model changes like that using the production data they have, so that exact change in all the metrics they care about is known upfront. Turned out they are like me: deploy and see what breaks.

[−] facundo_dbx 36d ago
Author here :) We did have high-level metrics and expectations for how this change would behave, but a couple of factors made it much harder to reason about in practice that were happening in parallel.

Data in these systems moves slowly and with a lot of inertia, so the effects show up gradually and can lag behind the change itself. On top of that, the impact wasn’t uniform. Most of the overhead came from a small subset of volumes, so it took time to isolate what was actually driving the increase. These systems are hard to test at scale!

[−] dangus 36d ago
It’s a shame that such fantastic engineering work is buried behind a product with so many annoyances dictated by the marketing/revenue teams.

I wish Dropbox would make some kind of “classic edition” that removed annoyances from their desktop client.

Until then, I’m using Filen. It’s fine, I have some qualms with it but it runs on every platform including Linux, it’s affordable, and end to end encrypted.

[−] rkagerer 36d ago
I wish Dropbox would make some kind of “classic edition” that removed annoyances from their desktop client.

They absolute do!

https://help.dropbox.com/installs/simplified-desktop-applica...

Here's a screenshot:

https://i.imgur.com/7g2xRJP.png

It's just a non-intrusive little menu that lives on your system tray. No ads, nags, bloat or unwanted new "features" pushed onto you. It resembles their original software much more than it does the latest garbage to come out of that company.

The context menu shortcuts in File Explorer for Copy Link, Share, and View on Dropbox still work. Sync works. Most of the other crap is gone. It's great. It was so refreshing when it got installed. I would have left Dropbox by now without it.

[−] dangus 36d ago
Wow! It looks maybe a little hard to trust given that it’s clearly designed for older OSes but maybe I’ll play with it since my account is free tier anyway.

(I guess for Linux I could run the headless daemon, I think only the standard desktop experience is available)

[−] qingcharles 32d ago
Wow, it basically looks like it has all the good bits and all the jank taken out. Thank you.
[−] hs86 36d ago
Google recently increased storage from 2 TB to 5 TB on their $20 AI plan, while Dropbox is still stuck at 2 or 3 TB for their $12/$20 plans.

They moved from 1 TB to 2 TB in mid-2019, and I wonder if they ever plan to pass on any of the gains from the past seven years of technological advancements, or if those gains are simply being captured on their side while we keep paying the same.

[−] microtonal 36d ago
Aside from bad pricing and us wanting to move our data to servers owned by a European company, the thing that that bothered me the most as a (former) paying customer was the constant upsell pushes. Every time I’d log in to the web interface they would show ads in the web interface (including pop up dialogs) to try to move me to another plan.

I’m already paying 20 Euro per month. Leave me alone.

Good riddance.

[−] qingcharles 32d ago
I'm transitioning a lot of my more valuable stuff over from GDrive to Dropbox. It's too easy for something to take out your entire Goog acct and not be able to unlock it, for one. Secondly, their synchronisation is pretty poor and always getting snarled up. Dropbox will let you synchronize across a LAN too without having the other clients wait for it to appear on the cloud. And lastly, if you accidentally delete something off GDrive you are up shit creek. The process for undeleting something (which is a support call) is absolutely horrible. Dropbox at least will give you minimum 30 days to rewind.
[−] timmmmmmay 36d ago
are these "technological advancements" in storage in the room with us right now? because I'm looking at today's price per TB and it's higher than it was in 2020
[−] kingleopold 36d ago
did you calculate it with real inflation adjusted price? not the BS numbers in financial media, FED etc. Since 2020 unlimited printer, inflation is not few %.
[−] jeffbee 36d ago
The immutability of extents is dictated by their SMR hardware, I believe.
[−] bluedino 36d ago
Does Amazon ever publish similar articles about S3?
[−] jason_friman 42d ago
[dead]
[−] znnajdla 36d ago
All this talk about a tool that isn’t open source?