Improving storage efficiency in Magic Pocket, Dropbox's immutable blob store (dropbox.tech)

by laluser 21 comments 60 points
Read article View on HN

21 comments

[−] nopurpose 36d ago

> Last year, we rolled out a new service that changed how data is placed across Magic Pocket. The change reduced write amplification for background writes, so each write triggered fewer backend storage operations. But it also had an unintended side effect: fragmentation increased, pushing storage overhead higher. Most of that growth came from a small number of severely under-filled volumes that consumed a disproportionate share of raw capacity

Me thinking big corps with huge infrastructure bills meticulously model changes like that using the production data they have, so that exact change in all the metrics they care about is known upfront. Turned out they are like me: deploy and see what breaks.

[−] dangus 36d ago
It’s a shame that such fantastic engineering work is buried behind a product with so many annoyances dictated by the marketing/revenue teams.

I wish Dropbox would make some kind of “classic edition” that removed annoyances from their desktop client.

Until then, I’m using Filen. It’s fine, I have some qualms with it but it runs on every platform including Linux, it’s affordable, and end to end encrypted.

[−] hs86 36d ago
Google recently increased storage from 2 TB to 5 TB on their $20 AI plan, while Dropbox is still stuck at 2 or 3 TB for their $12/$20 plans.

They moved from 1 TB to 2 TB in mid-2019, and I wonder if they ever plan to pass on any of the gains from the past seven years of technological advancements, or if those gains are simply being captured on their side while we keep paying the same.

[−] jeffbee 36d ago
The immutability of extents is dictated by their SMR hardware, I believe.
[−] bluedino 36d ago
Does Amazon ever publish similar articles about S3?
[−] jason_friman 42d ago
[dead]
[−] znnajdla 36d ago
All this talk about a tool that isn’t open source?