docs/philosophy: Initial commit #1054

RichiH · 2018-06-10T11:52:06Z

This was surprisingly hard; I went through several iterations, tossing things away, re-adding some, and refactoring into different files. On the plus side, I think what I have is already relatively cleanly separated. design & patterns is a collection of half-finished thoughts and sentences with tons of duplicates. No use pushing those atm.

I would be especially interested in large conceptual comments and missing sections, less so in specific wording as that is bound to change, yet again, once there's more content.

Signed-off-by: Richard Hartmann <richih@richih.org>

brian-brazil

At a high level, this seems a bit all over the place. I'm not getting a clear sense of what we do/don't do, it feels more like reverse engineering things from certain design decisions but missing other decisions.

brian-brazil · 2018-06-10T12:43:41Z

content/docs/philosophy/goals.md

+they need to take action in order to prevent undesired system state.
+
+Thus, its most important function is to keep the pipeline of ingestion, rule
+evaluation, and alert hand-off working.


notification sending, we don't use the term "alert hand-off"

brian-brazil · 2018-06-10T12:44:19Z

content/docs/philosophy/goals.md

+operational patterns for different parts of our ecosystem:
+
+For Prometheus itself, this means running every instance as an island of data
+completely detached from every other instance, except for optional federation.


This exception is weird, and incomplete. I'd go more with how any dependenceis have understandable impact.

I removed the half sentence completely.

brian-brazil · 2018-06-10T12:44:40Z

content/docs/philosophy/goals.md

+completely detached from every other instance, except for optional federation.
+
+For Alertmanager on the other hand, it means the exact opposite: meshing all
+instances closely together, sharing knowledge about alerts and their


This is too vague. The important point is that it's AP rather than CP

Please expand those two.

https://en.wikipedia.org/wiki/CAP_theorem

That didn't click in this context, but yes, we can simply refer to Brewer. It might lead to quite a bit more explanation around why we chose which. I would then be tempted to pull this in the design decisions simply link to each other; if linking at all.

brian-brazil · 2018-06-10T12:45:45Z

content/docs/philosophy/goals.md

+
+## Simple operation
+
+Operation of Prometheus should be as easy and failure-tolerant as possible. We


This is waaaay too open for interpretation. "easy" means I turn it on and it magically works with no configuration - which of course is far for easy in practice.

I'd talk about well defined and limited interfaces that allow integration

s/easy/simple/ and added a section on dependencies.

brian-brazil · 2018-06-10T12:46:06Z

content/docs/philosophy/goals.md

+try to put required complexity into earlier phases, going through them less
+often and ideally still while under the control of a smaller subset of people.
+
+One example of this would be the preference of statically linked binaries over


This is just an artifact of how Go works, it's not a project philosophy.

brian-brazil · 2018-06-10T12:48:24Z

content/docs/philosophy/goals.md

+# Push-type system
+
+Prometheus is, and always will be, a pull-type system. We strongly believe that
+this makes operational sense in all but the very largest of scales.


This is incorrect, we believe it works at all scales.

Also both ways work at scale. The more salient point is that a lot of things we do only really work with push.

I would argue that Monarch-style mixed systems allow for even more scale. This is beyond this document's scope, though.

Also, I think you meant pull. It makes sense to expand on those things, though. will put in a TODO into patterns and link back to there.

I disagree, both can be scaled indefinitely. Also Monarch is more pull than push.

Yeah, there's no limits to scale with either approach, but there's the usual known pros+cons for both approaches, and we're quite married to the pull approach, as Prometheus is so designed around being in control of pulling and processing data when it wants.

I'd just say the first sentence and then link to a blog post (Julius' from a while back perhaps or the FAQ entry. To the vast majority of users this is a non-issue.

Yet many users continue to try and make it do push (and tend to be quite unhappy when told their approach won't work and/or be supported), so I think it's worth a few sentences.

brian-brazil · 2018-06-10T12:49:03Z

content/docs/philosophy/philosophy.md

+  still doing one thing: ingest metric data, do computations on it, and expose
+  it to other systems.
+2. Expect the output of every program to become the input to another, as yet unknown, program.
+  Today's lingua franca is HTTP endpoints, which are used by Prometheus


HTTP endpoints with JSON

But this isn't what we use everywhere. We've our own text format, and file_sd and yaml for config files.

That's why I didn't put JSON.

But we're also doing many things that aren't HTTP.

Is there something you would propose instead?

I'd remove it.

brian-brazil · 2018-06-10T12:49:57Z

content/docs/philosophy/philosophy.md

+  We ensure that master always builds, called Continuous Integration these days,
+  and we not afraid to replace whole sections of our codebase, e.g. our storage
+  engine.
+4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them.


I don't understand what you're saying here

It's a literal quote. I can shorten it to stop at the first comma, though. Would that make more sense to you?

The "unskilled" part is an artefact; back then it was common to assign many tasks to what were basically filing clerks. They had the IT knowledge you would expect them to have.

That's a bit out of the scope of the project, as we don't run an operations team. It certainly has nothing to do with the Unix philosophy.

Hm yeah, I think some of these points sound a bit too forced in the context of Prometheus. I do agree about the "do one thing well" (which I would rather phrase as "keep pieces as simple as possible") and "open interfaces" points and would focus on those, while abandoning the others. I'm not sure we need to explicitly try to tie things so ceremoniously to the Unix philosophy.

brian-brazil · 2018-06-10T12:50:48Z

content/docs/philosophy/philosophy.md

+  While this is an outdated way of stating the goal, automation where possible
+  is still one of the core characteristics any modern philosophy.
+
+## Embrace cloud-native technologies


This is not our philosophy, in fact I'd say this explicitly isn't our philosophy.

We're a technology that happens to work well for things that fall into the "cloud native" marketing term - but we also work just as well outside of that.

With that structure, I wanted to say that this works well with good operations, but re-reading, I can see how that got lost over time.

Would you think it better to remove this or to expand on how cloud-native is one of the many facets of proper operations?

I'd have a section saying that we work across many paradigms, but aren't going to do things just for the sake of one system that is weird/missing a feature.

+1. Maybe some (more?) ideal use cases? Although that might drift over time and become stale.

Rewritten completely & moved down.

brian-brazil · 2018-06-10T12:51:16Z

content/docs/philosophy/philosophy.md

+
+Prometheus is a project of convicted and passionate individuals. As we do not
+have a profit motive, nor quarterly projections, or any other requirement to
+meet arbitrary business requirements, we can foxus on getting things right. This


Vim stopped to set spell for my markdown files... I need to fix that.

RichiH · 2018-06-10T12:58:03Z

I agree that it still lacks a consistent thread, but I figured I'd rather get partial feedback at this point to align.

Signed-off-by: Richard Hartmann <richih@richih.org>

RichiH · 2018-06-10T13:28:39Z

Are there any major sections & content missing from philosophy and goals?

What I am missing, on purpose, are any cross-links, those will come last.

Anything we can reasonably push to a lower tier, we should push there.

brian-brazil · 2018-06-10T13:34:00Z

content/docs/philosophy/philosophy.md

+
+# Be open
+
+We will always put as much of our code, discussions, presentations, and other


I think you're missing an "as possible" here gramatically

brian-brazil · 2018-06-10T13:34:27Z

content/docs/philosophy/philosophy.md

+
+# Play well with others
+
+Prometheus is a project of convicted and passionate individuals. As we do not


I don't think convicted was the word you meant

...are you sure...?

brian-brazil · 2018-06-10T13:35:08Z

content/docs/philosophy/philosophy.md

+also means that we are free to suggest other implementations and projects if
+they are a better fit for a particular use-case.
+
+# Be inclusive


It seems weird to mix this in with the other technical stuff

Is there another place where this is mentioned? I feel like this needs to be somewhere?

Only indirectly in our governance: https://prometheus.io/governance/#values

Maybe if/once we have a general section for developers and getting into Prometheus contributions, it would be a better fit there?

This is very vision statementy. It has little to no bearing on the design decisions we make.

Should we keep it in there while we don't have a better place, yet?

Put in a TODO for now.

brian-brazil · 2018-06-10T13:35:56Z

content/docs/philosophy/goals.md

+## Keep dependencies clear and limited
+
+Any non-trivial system needs to integrate with other systems. To keep the
+resulting complexity low, we will always try to have the fewest interfaces


Maybe mention to make debugging/understandability easier?

I'm not seeing this change.

juliusv · 2018-06-10T16:15:41Z

Are there any major sections & content missing from philosophy and goals?

Have you intentionally omitted most of the points from #149 so far?

brian-brazil · 2018-06-10T17:14:50Z

content/docs/philosophy/goals.md

+label rewriting, and alert generation.
+
+
+# Non-Goals


I think this is where we put in stuff about assuming you already have CM/DNS/service database/machine database etc., tying back to laying/doing one thing well.

You mean to put in that have CMDB etc explicitly outside our scope? Isn't that obvious?

We get requests in this area fairly regularly.

Signed-off-by: Richard Hartmann <richih@richih.org>

brian-brazil · 2018-06-11T17:54:51Z

content/docs/philosophy/philosophy.md

+content as possible into a form and place which is accessible in the long term,
+free of charge.
+
+# Be opionated


I'm not sure which section it goes in here, but there should be something about not adding attractive nuisances: features that might be useful to experts for niche use cases. but would be misunderstood by and cause problems for the average user. Put another way, we don't leave chainsaws lying about for users to hurt themselves with.

brian-brazil · 2018-06-11T17:55:48Z

content/docs/philosophy/goals.md

+try to put required complexity into earlier phases, going through them less
+often and ideally still while under the control of a smaller subset of people.
+
+One example of this would be the preference of statically linked binaries over


This is still here, this is not a goal of the project.

brian-brazil · 2018-06-11T17:55:58Z

content/docs/philosophy/goals.md

+## Keep dependencies clear and limited
+
+Any non-trivial system needs to integrate with other systems. To keep the
+resulting complexity low, we will always try to have the fewest interfaces


I'm not seeing this change.

brian-brazil · 2018-06-11T17:56:53Z

content/docs/philosophy/goals.md

+evaluation, and alert notifications working.
+
+The second most important function is to give humans context about these alerts
+by allowing access to the most recent data Prometheus ingested


fullstop

I'd add something around data that's good enough to make useful engineering decisions, rather than 100% perfect data.

* fix inconsistencies in file names and heading * use 2nd level heading for all flags * fix typo in meta * revert file name change until later * revert file name * escape * character

roidelapluie

@RichiH if you still want this I have made some extra comments.

roidelapluie · 2020-02-13T09:48:05Z

content/docs/philosophy/goals.md

+title: Goals and Non-Goals
+sort_rank: 3
+---
+


I think we miss an introduction here

roidelapluie · 2020-02-13T09:50:53Z

content/docs/philosophy/goals.md

+
+For Alertmanager on the other hand, it means the exact opposite: meshing all
+instances closely together, sharing knowledge about alerts and their
+notifications.


I would not say that they are closely meshed, just that they are meshed, as they can lose their mesh by design. I would also then specify what happens when they can't mesh.

roidelapluie · 2020-02-13T09:59:55Z

content/docs/philosophy/goals.md

+
+# Event handling
+
+Prometheus is dealing with metrics. As such, it will never process and store


And we get into exemplars

roidelapluie · 2020-02-13T10:10:34Z

content/docs/philosophy/philosophy.md

+
+We will always put as much of our code, discussions, presentations, and other
+content as possible into a form and place which is accessible in the long term,
+free of charge.


should we add in the open, too?

roidelapluie · 2020-02-13T10:11:35Z

content/docs/philosophy/philosophy.md

+
+# Play well with others
+
+Prometheus is a project of convinced and passionate individuals. As we do not


maybe we should review this about the recent discussions on tsdb code reuse etc

RichiH · 2020-02-13T14:17:52Z

@roidelapluie I do, it's on my bucket lest, thanks!

beorn7 · 2020-03-17T13:52:49Z

@RichiH , this PR seems to get some love on the mailing list. I guess to revive it, it needs first an update from your side before it makes sense for reviewers to comment on details.

grobie · 2020-10-15T17:03:04Z

@RichiH Any updates here? I think it would be great to get this in in some form.

docs/philosophy: Initial commit

5292ccc

Signed-off-by: Richard Hartmann <richih@richih.org>

RichiH requested review from brian-brazil and juliusv June 10, 2018 12:00

RichiH self-assigned this Jun 10, 2018

brian-brazil reviewed Jun 10, 2018

View reviewed changes

docs/philosophy/: Feedback by @brian-brazil and typos

102961b

Signed-off-by: Richard Hartmann <richih@richih.org>

brian-brazil reviewed Jun 10, 2018

View reviewed changes

RichiH added 3 commits June 10, 2018 22:31

docs/philosophy/: More feedback by @brian-brazil

b61084f

Signed-off-by: Richard Hartmann <richih@richih.org>

docs/philosophy/: More feedback by @brian-brazil

ac67a8c

Signed-off-by: Richard Hartmann <richih@richih.org>

docs/philosophy/: More feedback, some refactoring

63ce534

Signed-off-by: Richard Hartmann <richih@richih.org>

brian-brazil reviewed Jun 11, 2018

View reviewed changes

roidelapluie reviewed Feb 13, 2020

View reviewed changes

beorn7 self-requested a review February 13, 2020 15:01

beorn7 removed their request for review August 8, 2024 13:04


		## Simple operation

		Operation of Prometheus should be as easy and failure-tolerant as possible. We


		# Be open

		We will always put as much of our code, discussions, presentations, and other


		# Play well with others

		Prometheus is a project of convicted and passionate individuals. As we do not


		# Event handling

		Prometheus is dealing with metrics. As such, it will never process and store


		# Play well with others

		Prometheus is a project of convinced and passionate individuals. As we do not

docs/philosophy: Initial commit #1054

Are you sure you want to change the base?

docs/philosophy: Initial commit #1054

Conversation

RichiH commented Jun 10, 2018

brian-brazil left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juliusv Jun 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamtur01 Jun 10, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RichiH commented Jun 10, 2018

RichiH commented Jun 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juliusv commented Jun 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

roidelapluie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RichiH commented Feb 13, 2020

beorn7 commented Mar 17, 2020

grobie commented Oct 15, 2020

juliusv Jun 10, 2018 •

edited

Loading

jamtur01 Jun 10, 2018 •

edited

Loading