Deploying Nextcloud on AWS ECS with Pulumi (github.com)
from joegarciar3a3294@lemmy.world to selfhosted@lemmy.world on 09 Aug 05:12
https://lemmy.world/post/34172325

i am not a devops engineer. i appreciate any critique or correction.

code: gitlab github

Deploying Nextcloud on AWS ECS with Pulumi

This Pulumi programme deploys a highly-available, cost-effective Nextcloud service on AWS Fargate with a serverless Aurora PostgreSQL database.

Deployment Option 1 (GitOps)

The first few items are high-level instructions only. You can follow the instructions from the hyperlinked web pages. They include the best practices as recommended by the authors.

  1. A Pulumi account. This is for creating a Personal Access Token that is required when provisioning the AWS resources.
  2. Create a non-root AWS IAM User called pulumi-user.
  3. Create an IAM User Group called pulumi-group
  4. Add the pulumi-user to the pulumi-group User Group.
  5. Attach the IAMFullAccess policy to pulumi-group. The IAMFullAccess allows your IAM User to add the remaining required IAM policies to the IAM User Group using the automation script later.
  6. Create an access key for your non-root IAM User.
  7. On your Pulumi account, go to Personal access tokens and create a token.
  8. Also create a password for the Aurora Database. You can use a password generator.
  9. Clone this repository either to your GitLab or GitHub.
  10. This works either on GitLab CI/CD or GitHub Actions. On GitLab, go to the cloned repository settings > Settings > Variables. On GitHub, go to the cloned repository settings > Secrets and variables > Actions > Secrets.
  11. Store the credentials from steps 6-8 as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, PULUMI_ACCESS_TOKEN, and POSTGRES_PASSWORD. These will be used as environment variables by the deployment script.
  12. On AWS Console, go to EC2 > Load Balancers. The DNS name is where you access the Nextcloud Web Interface to establish your administrative credentials.

[!NOTE] The automatic deployment will be triggered if there are changes made on the main.go, .gitlab-ci.yml, or the ci.yml file upon doing a git push. On main.go, you can adjust the specifications of the resources to be manifested. Notable ones are in lines 327, 328, 571, 572, 602, 603, 640.

Deployment Option 2 (Manual)

  1. Install Go, AWS CLI, and Pulumi.
  2. Follow steps 1-8 above.
  3. Add the required IAM policies to the IAM User Group to allow Pulumi to interact with AWS resources:
printf '%s\n' "arn:aws:iam::aws:policy/AmazonS3FullAccess" "arn:aws:iam::aws:policy/AmazonECS_FullAccess" "arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess" "arn:aws:iam::aws:policy/CloudWatchEventsFullAccess" "arn:aws:iam::aws:policy/AmazonEC2FullAccess" "arn:aws:iam::aws:policy/AmazonVPCFullAccess" "arn:aws:iam::aws:policy/SecretsManagerReadWrite" "arn:aws:iam::aws:policy/AmazonElasticFileSystemFullAccess" "arn:aws:iam::aws:policy/AmazonRDSFullAccess" | xargs -I {} aws iam attach-group-policy --group-name pulumi-group --policy-arn {}
  1. Add the environment variables.
export PULUMI_ACCESS_TOKEN="value" && export AWS_ACCESS_KEY_ID="value" && export AWS_SECRET_ACCESS_KEY="value" && export POSTGRES_PASSWORD="value"
  1. Clone the repository locally and deploy.
mkdir pulumi-aws && \
cd pulumi-aws && \
pulumi new aws-go && \
rm * && \
git clone https://gitlab.com/joevizcara/pulumi-aws.git . && \
pulumi up

Deprovisioning

pulumi destroy --yes

Local Testing

The Pulumi.aws-go-dev.yaml file contains a code block to use with Localstack for local testing.

Features

  1. Subscription-free application - Nextcloud is a free and open-source cloud storage and file-sharing platform.
  2. Serverless management - using Fargate and Aurora Serverless reduces infrastructure management.
  3. Reduced cost - can be scaled and as highly available as an AWS EKS cluster, but with cost lower per-hour.
  4. Go coding language - a popular language for cloud-native applications, eliminating syntax barriers for engineers.

Diagramme

#selfhosted

threaded - newest

loudwhisper@infosec.pub on 09 Aug 06:20 next collapse

Everyone is free to pick their poison, but I have to ask…why? What is the target audience here? This is a massively overkill architecture IMHO. Not to talk about the fact you now need 3 managed services (fargate, s3 and aurora at least) for a single self hosted tool, and that is being generous (not counting cloudwatch, ALBs, etc.).

  • Why do you need security groups to allow egress anywhere (or, at all)?
  • I would pin the image to a digest, rather than using latest.
  • what is the average monthly cost for this infra for you?
EncryptKeeper@lemmy.world on 10 Aug 16:28 collapse

Folks in IT. This is one of those “deploy something enterprise grade because you can” type of scenarios. It’s like asking why somebody would play a dry milsim game like Arma when Call of Duty exists. This will cost you more than a simple VPS on a platform but it wouldn’t exactly break the bank either.

loudwhisper@infosec.pub on 10 Aug 19:18 collapse

Well yeah, wouldn’t break the bank, but a conservative cost estimate (without considering network costs, for example, quite relevant for a data intensive app) would bring this setup to about $40/month. That is about 5 times more expensive than a VPC with 4x the resources.

OP said this is some sort of “enterprise self-hosting” solution, which I guess then kind of makes sense. For a company providing nextcloud as a service I would never vendor lock myself and let AWS take a huge chunk of my revenue forever, but I can imagine folks have different opinions.

EncryptKeeper@lemmy.world on 11 Aug 01:23 collapse

A $10VPS would not be sufficient for a heavily used multi-user Nextcloud instance, and it wouldn’t come with enough storage either.

You could cloud host this thing for less absolutely, but not a whole lot less. I have a Vultr VPS (cheaper than Digital Ocean, Linode, and other cheapo VPS providers) and all it does is reverse proxy and do some caching and it’s scraping by at a total of $24 a month. A $40 solution that’s more functional if not over-engineered for the difference in price equivalence to a Netflix subscription is not that huge a deal. Especially for someone who is doing this level of work for their job. That $16 is chump change.

loudwhisper@infosec.pub on 11 Aug 06:47 collapse

But the estimation is with each NC instance with half a CPU and 1GB of memory. This is a super conservative estimation, that doesn’t include anything besides a tiny Fargate deployment and Aurora instances.

Edit: fargate ($40/month), the tiniest Aurora instances at 20% utilization and with merely 50GB storage ($120/month). Missing s3, which will easily cost $50 in storage and transfer (for only a few TB), ALBs and network traffic, especially outbound (easily $50-100 depending on volumes).

This basic solution’s real cost is already between $150 and $300/month. I don’t know NC enough to understand volumes on DBs and all usage, but I assume that it’s going to be lots of data in and out (backups, media, etc.). —edit—

For a heavily used NC instance (assuming a company offering it as a service), the cost is going to become massive pretty fast.

Also, as I side note, if a company is offering NC as a service, but doesn’t manage a single piece of NC deployment… What is the company product? And most importantly, how are they going to make money when AWS is going to eat a linearly scalable chunk of their revenue forever?

rimu@piefed.social on 09 Aug 06:29 next collapse

One availability zone is enough. I am not convinced that Aurora is good value.

This seems like an ad for pulumi, whatever that is.

felbane@lemmy.world on 09 Aug 14:57 collapse

In my experience, Pulumi can best be described as a waste of time.

dan@upvote.au on 09 Aug 10:01 next collapse

This seems like overkill compared to just running it on a VPS and having a second VPS as a hot spare.

napkin2020@sh.itjust.works on 09 Aug 14:36 next collapse

No sane selfhoster should do this. This is far beyond being overkill.

EncryptKeeper@lemmy.world on 10 Aug 16:28 collapse

I’m pretty sure that’s the point.

magikmw@piefed.social on 09 Aug 15:21 next collapse

All comments about overkill are amusing. You do you. Did you learn stuff?

Maybe you can replace some of those tools with less expensive analogs, how's the cost anyway?

joegarciar3a3294@lemmy.world on 09 Aug 15:45 collapse

it’s just for my portfolio. it’s like self-hosting for enterprise

loudwhisper@infosec.pub on 09 Aug 19:29 next collapse

In that case, Pulumi permissions are too broad IMHO for what it has to do, an enterprise should adhere to least privilege. Likewise, as I wrote in another comment, the egress security groups are unclear to me (why any traffic at all is needed?) and the image consumed should be pinned to a digest. Or better yet, should be coming from a private enterprise registry, ideally with an attestation that can be verified at runtime.

I am not sure ECS Fargate makes sense vs an ec2 instance to run the workload. This setup alone will cost about $30/month assuming half a vCPU per replica with Fargate, plus about $12 for the memory (1GB/task). 2xt2.micro could be run for ~$20 without even considering reservation discounts etc. Obviously the gap will become even larger at scale, which I suppose might be very interesting for an enterprise.

aurelian@lemmy.ml on 09 Aug 21:51 collapse

If you want it to stand out don’t automate the compute and networking that’s so standardized these days that anyone can do it, Automated those IAM permissions.

I know that when hiring nothing gets me more excited about a candidate than them understanding how to securely bootstrap an environment.

joegarciar3a3294@lemmy.world on 12 Aug 06:39 collapse

it is actually possible to add non-root iam accounts using your root iam account. however, in order to do that, you first have to create an access key for your root user, which is not recommended by aws.
<img alt="" src="https://lemmy.world/pictrs/image/a4de522a-1fd6-4983-b573-294a55af1086.png">

the method is to manually create a non-root user, add an initial iam permission to modify the iam resources, and automate the additional iam permission agregation via script in gitlab-ci.yml

rollerbang@lemmy.world on 09 Aug 19:42 next collapse

<img alt="" src="https://lemmy.world/pictrs/image/b3b28059-e6f4-414c-a183-f42b1e33bb69.jpeg">

e8d79@discuss.tchncs.de on 09 Aug 19:57 next collapse

Pretty cool as a learning exercise. As a follow up scenario maybe try moving this infrastructure to another cloud provider because AWS deleted your account without warning or try a multi-cloud deployment.

fubarx@lemmy.world on 10 Aug 06:15 collapse

A few suggestions:

  • Some of those components may end up costing a lot to operate. You said you’re doing it as a portfolio piece. May want to create a spreadsheet with all the services, then run a cost simulation. You can use the AWS Cost calculator, but it won’t be as flexible doing ‘what if’ scenarios. Any prospective employer will appreciate that you’ve given some thought to runtime pricing.

  • You may want to bifurcate static media out and put them in S3 buckets, plus put a CloudFront CDN in front for regional scaling (and cost). Static media coming out of local server uses up processing power, bandwidth, storage, and memory. S3/CloudFront is designed for just this and is a lot cheaper. All fonts, js scripts, images, CSS stylesheets, videos, etc. can be moved out.

  • Definitely expire your CloudWatch log records (maybe no more than a week), otherwise they’ll pile up and end up costing a lot.

  • Consider where backups and logs may go. Backups should also account for Disaster Recovery (DR). Is the purpose of multiple AZs for scaling or DR? If for DR, you should think about different recovery strategies and how much down-time is acceptable.

  • Using Pulumi is good if the goal is to go multi-cloud. But if you’ve hardcoded Aurora or ALBs into the stack, you’re stuck with AWS. If that’s the case, maybe consider going with AWS CDK in a language you like. It would get you farther and let you do more native DevOps.

  • Consider how updates and revisions might work, especially once rolled out. What scripts will you need to run to upgrade the NextCloud stack. What are the implications if only one AZ is updated, but not the other. Etc.

  • If this is meant for business or multiple users, consider where user accounts would go? What about OAuth or 2FA? If it’s a business, they may already have an Identity Provider (IDP) and now you need to tie into it.

  • If tire-kicking, may want to also script switching to plain old RDS/Postgres so you can stay under the free tier.

  • To make this all reusable, you want to take whatever is generated (i.e. Aurora endpoints, and save everything to a JSON or .env file. This way, the whole thing can be zapped and re-created and should work without having to manually do much in the console or CLI.

  • Any step that uses the console or CLI adds friction and risk. Either automate them, or document the crap out of them as a favor to your future self.

  • All secrets could go in .env files (which should be in .gitignore). Aurora/RDS Database passwords could also be auto-generated and kept in SecretsManager and periodically rotated. Hardcoded DB passwords are a risk.

  • Think about putting WAF in front of everything with web access to prevent DDOS attacks.

This is a great, learning exercise. Hope you don’t find these suggestions overwhelming. They only apply if you want to show it off for future employers. If it’s just for personal use, ignore all the rest I said and just think about operating costs. See if you can find an AWS sales or support person and get some freebie credits.

Best of luck!

EncryptKeeper@lemmy.world on 10 Aug 16:30 next collapse

This is an awesome helpful comment

joegarciar3a3294@lemmy.world on 10 Aug 17:10 next collapse

thank you

MysteriousSophon21@lemmy.world on 12 Aug 07:44 collapse

The S3 suggestion for static content is gold - I’ve seen this cut costs by 60-70% in similar setups since Fargate billing is mostly about runtime, not just storaeg.