Yeah, everyone struggles with that, especially since AWS doesn’t really break down costs in a way that makes sense if you’re trying to work out which business unit or feature is costing you money (unless you set things up where every team has their own account or whatever, and even then).
I think Netflix could probably start selling their tool and make more money from that then streaming if they can get it to work, because I’m guessing AWS makes it hard on purpose.
custard_swollower@lemmy.world
on 18 Dec 09:28
nextcollapse
Not sure what is hard in it - you need consistent tagging, and that by itself gives you a lot of mileage in cost explorer.
Not only is that free, but I can’t imagine a better alternative. And they would have the same issue with allocation on prem. WithOUT tagging and Cost Explorer.
BlueBockser@programming.dev
on 18 Dec 16:15
nextcollapse
That doesn’t work in all cases. I’ve recently come across two examples where we had a hard time explaining our costs even though we extensively tag and even have fine-grained AWS accounts:
Some costs can’t be tagged or at least not easily, e.g. custom CloudWatch metrics.
For some resources it makes a lot more sense to provide them centrally for multiple services at once, e.g. NAT gateways or load balancers.
There’s a lot of overlap of features. For example, we have a serverless function that does one thing for a dozen different departments, as well as for affiliates. Right now, our solution is just to say its part of the “operating expenses” bucket, which is our black box of things that just need to be paid.
The solution to fix this would be to split the function as separate instances, to get cleaner tag data.
But now you have separate instances of the same code and everything is even less efficient/effective, and significantly more overhead for “cleaner” tagging.
The serverless function is a extreme tiny example. Multiply that by bigger projects/critical architecture/etc.
For our company, we accept not really knowing the exact details, because the cost of getting that clarity is a magnitude more work for little gain.
custard_swollower@lemmy.world
on 20 Dec 09:21
collapse
My take is that it’s already your systems feature, rather than admins responsibility. If you treat departments like customers, you’d find a good way to spread the costs. If something is just a „common infrastructure”, you will always find something that makes costs that doesn’t have an easy way to track who triggered that - because you don’t pass enough information with it.
GamingChairModel@lemmy.world
on 18 Dec 11:50
collapse
Yeah, the actual headline to the article is “Even Netflix struggles to identify and understand the cost of its AWS estate,” which OP has very unhelpfully shortened to post “Netflix struggles to understand its cloud costs.”
The word “Even” is doing a lot of work, and leaving it out changes the meaning of the headline.
Also, other than the headline, you know, the article itself :)
GamingChairModel@lemmy.world
on 18 Dec 14:35
collapse
Of course. But even for people who don’t read the article, it’s still best practice to just copy the headline from the article so that it’s the top of the Lemmy thread.
IphtashuFitz@lemmy.world
on 18 Dec 13:42
nextcollapse
I think Netflix could probably start selling their tool and make more money from that then streaming if they can get it to work, because I’m guessing AWS makes it hard on purpose.
The company Cloudhealth Technologies has already been doing this for years, and not just with AWS but other cloud providers as well. Unfortunately they’ve been acquired by VMware so no idea if they’re still as good as they used to be…
This just isn’t true. AWS is very clear about pricing and provides pretty good tooling for analysis. Complex infrastructure will have complex costs though. AWS has its problems a’plenty, but unclear costs isn’t one in my opinion.
I agree, works fine for my stuff, but I’m pretty heavy on vanilla ec2, RDS, ec.
I can see some people running some of the more complicated products and getting flustered. Some of the compound stuff where you’re paying extra transaction costs on top of the serverless things
Not to mention the costs change on the fly since load changes on the fly.
And yea AWS is like a used car dealer, they want the TCOS to be hard to calculate … “Oh you want the car to have wheels. That’s an extra fee…”.
Passerby6497@lemmy.world
on 18 Dec 13:48
nextcollapse
Lol, is Netflix any of my customers who thought moving to public cloud services was a good idea?
I would ask why customers are so dumb about how much public cloud offerings cost, but I know it’s a combination of 1) seemingly low prices up front, 2) bean counters that aren’t able to accurately cost forecast long term, and 3) a fundamental understanding of just how much compute they actually need. It’s absolutely stupid how many customers we have that are having to pull back on services because some genius thought moving to azure was a good financial move only to find out 6-8mo later that they’ve spent this year’s budget already.
billwashere@lemmy.world
on 19 Dec 16:06
nextcollapse
Thank you. I’ve never understood the desire to implement bistromathics into my yearly costs. Since it requires a degree in fortune telling and an advanced degree in math to determine what it’s actually going to cost, I don’t want that complexity in my budget. I want to buy hardware and licenses and know that it will cost $X (well unless Broadcom and VMWare licenses are involved).
I’m struggling to understand how a company as big as Netflix is paying for a cloud service. Like for the cost that they’re charging customers monthly on top of how big they are, I really figured they would be running their own infrastructure at this point it seems like a needless money Leach to not be. Like sure you have to pay for the infrastructure and maintaining the infrastructure, but there is no way that on the scale that Netflix is and with how data transfer heavy it is, that it’s more cost-effective to be running a cloud stack.
Netflix has a substantial number of caching boxes owned by Netflix on the edge.
This means that a neighborhood has a cache server with 20TB of video serving everyone in the neighborhood (and maybe also the next 3 or 4 neighborhoods). So if everyone watches the same episode they only pay for one cloud download / bandwidth cost.
Netflix outsources cloud for their core infra but then runs very very heavy edges around the world. It’s backwards but if you think about Netflix and the watching pattern of neighborhoods (ex: neighbor likely tells friends a new show is good, then everyone watches it) it’s to the benefit of Netflix to run their infra like this.
I wouldn’t be surprised if these caching servers help each other out like BitTorrent either to avoid central cloud bandwidth costs.
UntitledQuitting@reddthat.com
on 19 Dec 20:37
collapse
threaded - newest
The author has no clue how spending works in cloud environments nor why it’s so complicated to calculate. This is a pretty uniformed article.
Yeah, everyone struggles with that, especially since AWS doesn’t really break down costs in a way that makes sense if you’re trying to work out which business unit or feature is costing you money (unless you set things up where every team has their own account or whatever, and even then).
I think Netflix could probably start selling their tool and make more money from that then streaming if they can get it to work, because I’m guessing AWS makes it hard on purpose.
Not sure what is hard in it - you need consistent tagging, and that by itself gives you a lot of mileage in cost explorer.
Not only is that free, but I can’t imagine a better alternative. And they would have the same issue with allocation on prem. WithOUT tagging and Cost Explorer.
That doesn’t work in all cases. I’ve recently come across two examples where we had a hard time explaining our costs even though we extensively tag and even have fine-grained AWS accounts:
There’s a lot of overlap of features. For example, we have a serverless function that does one thing for a dozen different departments, as well as for affiliates. Right now, our solution is just to say its part of the “operating expenses” bucket, which is our black box of things that just need to be paid.
The solution to fix this would be to split the function as separate instances, to get cleaner tag data.
But now you have separate instances of the same code and everything is even less efficient/effective, and significantly more overhead for “cleaner” tagging.
The serverless function is a extreme tiny example. Multiply that by bigger projects/critical architecture/etc.
For our company, we accept not really knowing the exact details, because the cost of getting that clarity is a magnitude more work for little gain.
My take is that it’s already your systems feature, rather than admins responsibility. If you treat departments like customers, you’d find a good way to spread the costs. If something is just a „common infrastructure”, you will always find something that makes costs that doesn’t have an easy way to track who triggered that - because you don’t pass enough information with it.
That’s the point of the article
Yeah, the actual headline to the article is “Even Netflix struggles to identify and understand the cost of its AWS estate,” which OP has very unhelpfully shortened to post “Netflix struggles to understand its cloud costs.”
The word “Even” is doing a lot of work, and leaving it out changes the meaning of the headline.
Also, other than the headline, you know, the article itself :)
Of course. But even for people who don’t read the article, it’s still best practice to just copy the headline from the article so that it’s the top of the Lemmy thread.
The company Cloudhealth Technologies has already been doing this for years, and not just with AWS but other cloud providers as well. Unfortunately they’ve been acquired by VMware so no idea if they’re still as good as they used to be…
This just isn’t true. AWS is very clear about pricing and provides pretty good tooling for analysis. Complex infrastructure will have complex costs though. AWS has its problems a’plenty, but unclear costs isn’t one in my opinion.
So Netflix built this tool just for fun then? Obviously they feel AWS’ tools are lacking.
.
I agree, works fine for my stuff, but I’m pretty heavy on vanilla ec2, RDS, ec.
I can see some people running some of the more complicated products and getting flustered. Some of the compound stuff where you’re paying extra transaction costs on top of the serverless things
Not to mention the costs change on the fly since load changes on the fly.
And yea AWS is like a used car dealer, they want the TCOS to be hard to calculate … “Oh you want the car to have wheels. That’s an extra fee…”.
Lol, is Netflix any of my customers who thought moving to public cloud services was a good idea?
I would ask why customers are so dumb about how much public cloud offerings cost, but I know it’s a combination of 1) seemingly low prices up front, 2) bean counters that aren’t able to accurately cost forecast long term, and 3) a fundamental understanding of just how much compute they actually need. It’s absolutely stupid how many customers we have that are having to pull back on services because some genius thought moving to azure was a good financial move only to find out 6-8mo later that they’ve spent this year’s budget already.
Thank you. I’ve never understood the desire to implement bistromathics into my yearly costs. Since it requires a degree in fortune telling and an advanced degree in math to determine what it’s actually going to cost, I don’t want that complexity in my budget. I want to buy hardware and licenses and know that it will cost $X (well unless Broadcom and VMWare licenses are involved).
2b) bean counters who are ordered by middle managers to prioritize short term profits over all other considerations.
bean countersmiddle managers who are ordered bymiddle managersshareholders to prioritize short term profits over all other considerations.Why? They also using AWS?
They do, actually!
I struggle to understand my Netflix costs.
I’m struggling to understand how a company as big as Netflix is paying for a cloud service. Like for the cost that they’re charging customers monthly on top of how big they are, I really figured they would be running their own infrastructure at this point it seems like a needless money Leach to not be. Like sure you have to pay for the infrastructure and maintaining the infrastructure, but there is no way that on the scale that Netflix is and with how data transfer heavy it is, that it’s more cost-effective to be running a cloud stack.
Netflix has a substantial number of caching boxes owned by Netflix on the edge.
This means that a neighborhood has a cache server with 20TB of video serving everyone in the neighborhood (and maybe also the next 3 or 4 neighborhoods). So if everyone watches the same episode they only pay for one cloud download / bandwidth cost.
Netflix outsources cloud for their core infra but then runs very very heavy edges around the world. It’s backwards but if you think about Netflix and the watching pattern of neighborhoods (ex: neighbor likely tells friends a new show is good, then everyone watches it) it’s to the benefit of Netflix to run their infra like this.
I wouldn’t be surprised if these caching servers help each other out like BitTorrent either to avoid central cloud bandwidth costs.
Oh so Netflix is a debrid. Good to know