Improve load-balancer compatibility: Remove local FS storage requirement

[timkelty]

Running Craft on a load-balancer can be a tedious task because of its reliance on a single local file storage (CRAFT_STORAGE_PATH).

We can manage assets on s3 and store sessions in redis, but we're still left with the few things that Craft uses (complied cp resources, cp rebrand assets, etc)

I want to avoid sticky sessions and a shared mounted volume (e.g. NFS) because performance will suffer.

Ideally, I'd be able to use S3 for all storage, but I'm open to other solutions.

Related:

• Allow CP rebrand assets to be Asset element selection fields #2584
• https://gist.github.com/khalwat/a0687a8a8f0bf3ef5db7c371d88d3e04#gistcomment-2375152
• https://craftcms.stackexchange.com/a/4729

[angrybrad]

Ideally, I'd be able to use S3 for all storage

Probably not the best if you're worried about performance? Log files getting flushed at the end of every request to S3 would add a bit of overhead.

[noahjahn]

Removing local FS storage requirement would be awesome. Assuming I'm using a logging service, why not introduce external storage for compiled templates and other runtime files?

I'm hosting a craft site in docker containers on a platform that provides autoscaling, as long as I don't need to persist or share data on the local file system of the containers.

[timkelty]

@noahjahn I'm not sure of the real-world benefit of relocating compiled templates / runtime to an external resource. My guess is the overhead of the inherent network connection wouldn't be any faster than just have those regenerated on every host in your scaling group.

Would be interesting to test, though.

That all said, the approach we're moving toward is leaving this decision up to the developer, for each of these locations individually (eg not just storage).

[timkelty]

Probably not the best if you're worried about performance

Indeed :), I suppose I was referring to assets that get served (cpresources).
Temp/runtime things I'd still want on the filesystem, per-server.

[ostark]

@timkelty Maybe this is something for you: https://github.com/fortrabbit/craft-asset-bundler/blob/master/CHANGELOG.md
The Plugin is still WIP and requires more testing. Feel free to give it a try (dev-master) and provide feedback.

For logs you should use a centralized logging services, like Papertrail or Sentry

[timkelty]

Yep - already using Papertrail/AWS CloudWatch Logs agent for logs

Thanks @ostark, I'll check it out the plugin.
@angrybrad any interest in a first party solution here?