Feedback on Installation / Getting Started

[vsoch]

Here is some early feedback on your documentation. First - I'd recommend having it rendered via readthedocs (or sphinx locally) and remove it from your branded website. The format of this site and navigation is hard to read, and it's distracting to be alongside your profile, about, and basically many prompts to ask for funding. The project should stand alone on its own site. For example, for our flux projects we use a GitHub pages rendered mkdocs template (e.g.,).

For your blog post, in response to this line:

I was pleasantly surprised at how close each of the individual simulation runs were to each other.

I think this reflects your sampling. A "real world" collection of data from, for example, an autoscaler, is going to vary between times that you run it. Typically for experiments we will do multiple iterations of these real world runs, and then generate samples using that. There should be some variation in the individual simulation runs because we know there is in real clusters. It's a red flag that there is not. My suggestion is to run the real studies a few times to assess this and then have the simulation reflect that.

For the first part of the tutorial you tell the reader to git clone, but you don't give them the repository or a command. I think that the tutorial should have the git clone - people generally will want to copy paste it.

For the make build it's not clear to the reader that cargo is required. make build will likely fail for a large subset. You should explain that / link to instructions to install it. Here is what happened for me:

I saw it coming because I looked at the Makefile, but ran it anyway to show you the result.

For this part (the end of installation): https://appliedcomputing.io/simkube/docs/intro/installation/#customizing-simkube
You don't give the reader any context about where/when to apply these. It's not clear what is optional vs. required, and what files I'd write and how to apply. It's not clear if they are entire files or pieces. If this is a getting started / install, my desire would be for it to walk me through a full setup, be clear about what I need to do, and then provide the full files. My instinct is to do nothing, but maybe I'll end up with a borked tool?

I also have no idea where to go at the end of this page - it just stops. If I go to what I think is the next section:

https://appliedcomputing.io/simkube/docs/intro/running/

I'm supposedly supposed to start with having pods in the simkube namespace? I do not, and I'm not sure how they get there. So this is my stopping point for now. Let me know if/when you get a chance to update, thank you!

[drmorr0]

Hi, @vsoch, thanks so much for your feedback! I really appreciate it.

Website Improvements

I don't love the documentation site right now, and agree there are some improvements that need to be made to it. However, I am a bit puzzled by

basically many prompts to ask for funding.

I do not think there are... any prompts for funding on any of the documentation pages? If there are, can you please point them out to me? These definitely don't belong in the docs.

Blog post comment

You said,

There should be some variation in the individual simulation runs because we know there is in real clusters.

Yes, absolutely agree. My point in the first part of the blog post is: given the same trace file (input) the simulator should give the same output, which is exactly what we observe. If it doesn't, that's a problem. In the second part of the series you can see the variation that you're expecting.

Installation steps

1. I've added a line to git clone that people can copy.
2. Regarding "cargo being required" -- the very first section of the installation docs states that Rust is a prerequisite. I've updated this to say "including cargo", and added links to the "getting started" pages for the various prereqs.
3. I've also clarified that running make build, etc., is the expected path for developers, not for users. Users "should" be able to just use the provided kustomize files; can you let me know if those don't work for you? Or is there some other reason you wanted to build from source instead of using the provided images?
4. One step that was missing previously was installation of skctl; this is now on crates.io so you can run cargo install skctl instead of building from the repo. I've also updated the docs to indicate this.
5. I've updated the docs to indicate where the tracer config is injected into the sk-tracer pod, and to mention that you shouldn't need to mess with the customization options to get started. Ideally this section would go on a different page, but that's a project for another time.
6. For next steps:

I'm supposedly supposed to start with having pods in the simkube namespace?

If you apply the provided kustomize manifests, this should create the pods for you. If you are not using kustomize, you need to run make image and make run as described in the installation page -- did you do these steps? Was there any output or error messages? Was the simkube namespace created, and does it have a Deployment in it?

[drmorr0]

@vsoch I've done a bit more re-organization of the docs; I've moved all the stuff intended for devs into the "developers" section, and moved the customization/configuration into its own page as well. Hopefully this is a bit clearer now!

[hamzalsheikh]

I am also going through the installation part of the documentation. I would say it is fairly straight forward to setup the simulation cluster. However, I couldn't find any guidance in setting up sk-tracer in the production cluster. Applying the provided kustomize manifests create the sk-tracer pod in the simulation cluster which if I understood correctly should not be the right thing to do but I might be wrong.

Also, it would be nice to have a part about the prometheus setup/visualization, this is part of having a holistic getting started/installation section. (I followed your recent blog posts and maybe having aspects of that in the getting started section would be a good starter)

I agree with @vsoch that I expect this section to get me up and running with the full lifecycle of using simkube.

Great work btw! (Do let me know if I'm cluttering this feedback, I can start my own).

[drmorr0]

@hamzalsheikh Thanks for your kind words; happy to keep feedback in here. I converted this to a discussion so it's a bit easier to track different threads.

As you point out, there is an issue in the docs about "where to install sk-tracer vs. sk-ctrl" -- you are correct that they will just both get installed in the same place. I've been thinking about the best way to make this process easier, the intention is that sk-tracer gets installed in your production cluster and sk-ctrl gets installed in your simulation cluster. As a short-term workaround, you can just apply the sk-tracer yaml to your production environment, and all of the other resources in k8s/kustomize can be applied in your simulation cluster.

I'm thinking maybe an install script is going to be helpful here? I may invest some time into writing a basic one.

[drmorr0]

@hamzalsheikh @vsoch FYI I've split out the kustomize YAML for prod and your simulation environments, and updated the docs so it's easier to tell how to install which where. :P