From 5292ccc5705e151418f361cd9ed98484fd0ef8d5 Mon Sep 17 00:00:00 2001 From: Richard Hartmann Date: Sun, 10 Jun 2018 13:47:24 +0200 Subject: [PATCH 1/5] docs/philosophy: Initial commit Signed-off-by: Richard Hartmann --- content/docs/philosophy/goals.md | 72 +++++++++++++++++++ content/docs/philosophy/index.md | 5 ++ content/docs/philosophy/philosophy.md | 69 ++++++++++++++++++ .../docs/philosophy/philosophy_overview.md | 16 +++++ 4 files changed, 162 insertions(+) create mode 100644 content/docs/philosophy/goals.md create mode 100644 content/docs/philosophy/index.md create mode 100644 content/docs/philosophy/philosophy.md create mode 100644 content/docs/philosophy/philosophy_overview.md diff --git a/content/docs/philosophy/goals.md b/content/docs/philosophy/goals.md new file mode 100644 index 000000000..72c9ce005 --- /dev/null +++ b/content/docs/philosophy/goals.md @@ -0,0 +1,72 @@ +--- +title: Goals and Non-Goals +sort_rank: 3 +--- + +# Goals + +## Resilience + +First and foremost, Prometheus must be resilient in operation. + + +## Reliable alerting + +As a monitoring system, Prometheus is being relied upon to alert humans that +they need to take action in order to prevent undesired system state. + +Thus, its most important function is to keep the pipeline of ingestion, rule +evaluation, and alert hand-off working. + +The second most important function is to give humans context about these alerts +by allowing access to the most recent data Prometheus ingested + +### Resulting design decisions and patterns + +Note that this goal might result in widely different design decisions and thus +operational patterns for different parts of our ecosystem: + +For Prometheus itself, this means running every instance as an island of data +completely detached from every other instance, except for optional federation. + +For Alertmanager on the other hand, it means the exact opposite: meshing all +instances closely together, sharing knowledge about alerts and their +notifications. + +## Simple operation + +Operation of Prometheus should be as easy and failure-tolerant as possible. We +try to put required complexity into earlier phases, going through them less +often and ideally still while under the control of a smaller subset of people. + +One example of this would be the preference of statically linked binaries over +dynamically-built ones. + +## Automation + +Computers are good at doing the same thing over and over again, and quickly. +Humans tend to be better at creative tasks. + +Prometheus will always strive to automate away all tasks whenever possible +through various means; some specific implementations would be service discovery, +label rewriting, and alert generation. + + +# Non-Goals + +# Event handling + +Prometheus is dealing with metrics. As such, it will never process and store +events. + +The only exception in our ecosystem is Alertmanager which deals with individual +alerts and alert groups. + +For ways to deal with events, see TODO patterns. + +# Push-type system + +Prometheus is, and always will be, a pull-type system. We strongly believe that +this makes operational sense in all but the very largest of scales. + +For ways to integrate with push-type systems, see TODO patterns. diff --git a/content/docs/philosophy/index.md b/content/docs/philosophy/index.md new file mode 100644 index 000000000..d95f16ace --- /dev/null +++ b/content/docs/philosophy/index.md @@ -0,0 +1,5 @@ +--- +title: Philosophy +sort_rank: 99 +nav_icon: flask +--- diff --git a/content/docs/philosophy/philosophy.md b/content/docs/philosophy/philosophy.md new file mode 100644 index 000000000..67856ac97 --- /dev/null +++ b/content/docs/philosophy/philosophy.md @@ -0,0 +1,69 @@ +--- +title: Philosophy +sort_rank: 2 +--- + +# Do one thing well + +We believe in the [Unix +philosophy](https://en.wikipedia.org/wiki/Unix_philosophy), abridged from [Doug +McIlroy's initial version from 1978](http://emulator.pdp-11.org.ru/misc/1978.07_-_Bell_System_Technical_Journal.pdf): + +1. Make each program do one thing well. + While the scope of "one thing" invariably encompasses more and more + elements due to increased overall system and computing complexity, we are + still doing one thing: ingest metric data, do computations on it, and expose + it to other systems. +2. Expect the output of every program to become the input to another, as yet unknown, program. + Today's lingua franca is HTTP endpoints, which are used by Prometheus + extensively. + In the same vein, Prometheus relies heavily on its own libraries and strict + layering internally. +3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them. + Prometheus will always be available for free as in beer and as in speech. + We ensure that master always builds, called Continuous Integration these days, + and we not afraid to replace whole sections of our codebase, e.g. our storage + engine. +4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them. + While this is an outdated way of stating the goal, automation where possible + is still one of the core characteristics any modern philosophy. + +## Embrace cloud-native technologies + +In many ways, the cloud-native approach mirrors the Unix philosphy, updating it +for the modern world. + +1. Micro-services are the equivalent of doing one thing well +2. Ubiquitous APIs enable interoperatbility in the cloud-native world +3. Releasing early, realeasing often, and failing quickly is important when +failure is part of expected operations +4. Automation is key, freeing up humans to make more useful use of their time + +# Be pragmatic + +To not lose focus, we need to be honest to our users and ourselves about what we +can do and not do. + +# Be open + +We will always put as much of our code, discussions, presentations, and other +content into a form and place which is accessible in the long term, free of +charge. + +# Play well with others + +Prometheus is a project of convicted and passionate individuals. As we do not +have a profit motive, nor quarterly projections, or any other requirement to +meet arbitrary business requirements, we can foxus on getting things right. This +also means that we are free to suggest other implementations and projects if +they are a better fit for a particular use-case. + +# Be inclusive + +We strongly believe that technology should be accessible to all. As such, we +will always strive to be welcoming to everyone. + +As an example of this, many of us are investing their personal time helping +individuals or communities by educating and helping them to be more productive +in the tech sector, as well as sponsoring diversity efforts, for example paying +for travel and accomodation at [PromCon](https://promcon.io). diff --git a/content/docs/philosophy/philosophy_overview.md b/content/docs/philosophy/philosophy_overview.md new file mode 100644 index 000000000..d1eaca71e --- /dev/null +++ b/content/docs/philosophy/philosophy_overview.md @@ -0,0 +1,16 @@ +--- +title: Philosophy overview +sort_rank: 1 +--- + +We have structured our considerations for developing and working with Promethues +into four groups in decreasing order of importance, and likelihood to change. + +1. Philosophy +2. Goals and Non-Goals +3. Design decisions +4. Patterns and Anti-Patterns + +While we are not implying that anything in this section is likely to change at +all, it's still more likely for us to change a particular pattern over time than +the underlying philosophy. From 102961bb9f608b3d7119978843e9628bb284a51f Mon Sep 17 00:00:00 2001 From: Richard Hartmann Date: Sun, 10 Jun 2018 15:21:11 +0200 Subject: [PATCH 2/5] docs/philosophy/: Feedback by @brian-brazil and typos Signed-off-by: Richard Hartmann --- content/docs/philosophy/goals.md | 12 +++++++++--- content/docs/philosophy/philosophy.md | 10 +++++----- content/docs/philosophy/philosophy_overview.md | 2 +- 3 files changed, 15 insertions(+), 9 deletions(-) diff --git a/content/docs/philosophy/goals.md b/content/docs/philosophy/goals.md index 72c9ce005..17ac20b56 100644 --- a/content/docs/philosophy/goals.md +++ b/content/docs/philosophy/goals.md @@ -16,7 +16,7 @@ As a monitoring system, Prometheus is being relied upon to alert humans that they need to take action in order to prevent undesired system state. Thus, its most important function is to keep the pipeline of ingestion, rule -evaluation, and alert hand-off working. +evaluation, and alert notifications working. The second most important function is to give humans context about these alerts by allowing access to the most recent data Prometheus ingested @@ -27,7 +27,7 @@ Note that this goal might result in widely different design decisions and thus operational patterns for different parts of our ecosystem: For Prometheus itself, this means running every instance as an island of data -completely detached from every other instance, except for optional federation. +completely detached from every other instance. For Alertmanager on the other hand, it means the exact opposite: meshing all instances closely together, sharing knowledge about alerts and their @@ -35,13 +35,19 @@ notifications. ## Simple operation -Operation of Prometheus should be as easy and failure-tolerant as possible. We +Operation of Prometheus should be as simple and failure-tolerant as possible. We try to put required complexity into earlier phases, going through them less often and ideally still while under the control of a smaller subset of people. One example of this would be the preference of statically linked binaries over dynamically-built ones. +## Keep dependencies clear and limited + +Any non-trivial system needs to integrate with other systems. To keep the +resulting complexity low, we will always try to have the fewest interfaces +possible and keep their resulting complexity as low as possible. + ## Automation Computers are good at doing the same thing over and over again, and quickly. diff --git a/content/docs/philosophy/philosophy.md b/content/docs/philosophy/philosophy.md index 67856ac97..38f9244ff 100644 --- a/content/docs/philosophy/philosophy.md +++ b/content/docs/philosophy/philosophy.md @@ -30,12 +30,12 @@ McIlroy's initial version from 1978](http://emulator.pdp-11.org.ru/misc/1978.07_ ## Embrace cloud-native technologies -In many ways, the cloud-native approach mirrors the Unix philosphy, updating it +In many ways, the cloud-native approach mirrors the Unix philosophy, updating it for the modern world. 1. Micro-services are the equivalent of doing one thing well -2. Ubiquitous APIs enable interoperatbility in the cloud-native world -3. Releasing early, realeasing often, and failing quickly is important when +2. Ubiquitous APIs enable interoperability in the cloud-native world +3. Releasing early, releasing often, and failing quickly is important when failure is part of expected operations 4. Automation is key, freeing up humans to make more useful use of their time @@ -54,7 +54,7 @@ charge. Prometheus is a project of convicted and passionate individuals. As we do not have a profit motive, nor quarterly projections, or any other requirement to -meet arbitrary business requirements, we can foxus on getting things right. This +meet arbitrary business requirements, we can focus on getting things right. This also means that we are free to suggest other implementations and projects if they are a better fit for a particular use-case. @@ -66,4 +66,4 @@ will always strive to be welcoming to everyone. As an example of this, many of us are investing their personal time helping individuals or communities by educating and helping them to be more productive in the tech sector, as well as sponsoring diversity efforts, for example paying -for travel and accomodation at [PromCon](https://promcon.io). +for travel and accommodation at [PromCon](https://promcon.io). diff --git a/content/docs/philosophy/philosophy_overview.md b/content/docs/philosophy/philosophy_overview.md index d1eaca71e..f11a5a28d 100644 --- a/content/docs/philosophy/philosophy_overview.md +++ b/content/docs/philosophy/philosophy_overview.md @@ -3,7 +3,7 @@ title: Philosophy overview sort_rank: 1 --- -We have structured our considerations for developing and working with Promethues +We have structured our considerations for developing and working with Prometheus into four groups in decreasing order of importance, and likelihood to change. 1. Philosophy From b61084fcf00309cd36c04da3833de8ae865a9c59 Mon Sep 17 00:00:00 2001 From: Richard Hartmann Date: Sun, 10 Jun 2018 22:31:01 +0200 Subject: [PATCH 3/5] docs/philosophy/: More feedback by @brian-brazil Signed-off-by: Richard Hartmann --- content/docs/philosophy/philosophy.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/docs/philosophy/philosophy.md b/content/docs/philosophy/philosophy.md index 38f9244ff..5521efff6 100644 --- a/content/docs/philosophy/philosophy.md +++ b/content/docs/philosophy/philosophy.md @@ -47,12 +47,12 @@ can do and not do. # Be open We will always put as much of our code, discussions, presentations, and other -content into a form and place which is accessible in the long term, free of -charge. +content as possible into a form and place which is accessible in the long term, +free of charge. # Play well with others -Prometheus is a project of convicted and passionate individuals. As we do not +Prometheus is a project of convinced and passionate individuals. As we do not have a profit motive, nor quarterly projections, or any other requirement to meet arbitrary business requirements, we can focus on getting things right. This also means that we are free to suggest other implementations and projects if From ac67a8c7f358bb00982816305057507c62f88317 Mon Sep 17 00:00:00 2001 From: Richard Hartmann Date: Sun, 10 Jun 2018 22:34:53 +0200 Subject: [PATCH 4/5] docs/philosophy/: More feedback by @brian-brazil Signed-off-by: Richard Hartmann --- content/docs/philosophy/goals.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/docs/philosophy/goals.md b/content/docs/philosophy/goals.md index 17ac20b56..d8e46f8dc 100644 --- a/content/docs/philosophy/goals.md +++ b/content/docs/philosophy/goals.md @@ -46,7 +46,8 @@ dynamically-built ones. Any non-trivial system needs to integrate with other systems. To keep the resulting complexity low, we will always try to have the fewest interfaces -possible and keep their resulting complexity as low as possible. +possible and keep their resulting complexity as low as possible. This makes +understanding the system and thus working on and with it easier. ## Automation From 63ce534c1c5f91d739c50567c3a9c81f1ee70fee Mon Sep 17 00:00:00 2001 From: Richard Hartmann Date: Mon, 11 Jun 2018 13:47:06 +0200 Subject: [PATCH 5/5] docs/philosophy/: More feedback, some refactoring Signed-off-by: Richard Hartmann --- content/docs/philosophy/goals.md | 4 +++- content/docs/philosophy/philosophy.md | 26 +++++++++++++------------- 2 files changed, 16 insertions(+), 14 deletions(-) diff --git a/content/docs/philosophy/goals.md b/content/docs/philosophy/goals.md index d8e46f8dc..198c588ff 100644 --- a/content/docs/philosophy/goals.md +++ b/content/docs/philosophy/goals.md @@ -74,6 +74,8 @@ For ways to deal with events, see TODO patterns. # Push-type system Prometheus is, and always will be, a pull-type system. We strongly believe that -this makes operational sense in all but the very largest of scales. +this makes operational sense for what we are trying to achieve. Prometheus will +never become a push-type system, but we will make a reasonable effort to +integrate with other push-type systems. For ways to integrate with push-type systems, see TODO patterns. diff --git a/content/docs/philosophy/philosophy.md b/content/docs/philosophy/philosophy.md index 5521efff6..469d3f28b 100644 --- a/content/docs/philosophy/philosophy.md +++ b/content/docs/philosophy/philosophy.md @@ -24,20 +24,15 @@ McIlroy's initial version from 1978](http://emulator.pdp-11.org.ru/misc/1978.07_ We ensure that master always builds, called Continuous Integration these days, and we not afraid to replace whole sections of our codebase, e.g. our storage engine. -4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them. - While this is an outdated way of stating the goal, automation where possible - is still one of the core characteristics any modern philosophy. +4. Use tools in preference to \[manual work\] + Automation where possible is still one of the core characteristics any modern + philosophy. -## Embrace cloud-native technologies +# Work well across a variety of paradigms -In many ways, the cloud-native approach mirrors the Unix philosophy, updating it -for the modern world. - -1. Micro-services are the equivalent of doing one thing well -2. Ubiquitous APIs enable interoperability in the cloud-native world -3. Releasing early, releasing often, and failing quickly is important when -failure is part of expected operations -4. Automation is key, freeing up humans to make more useful use of their time +There are a lot of paragims which fundamentally try to achieve clean design, +implementation, and operation. We are not bound to any single of them, but +Prometheus tends to work well with any good practice. # Be pragmatic @@ -50,6 +45,11 @@ We will always put as much of our code, discussions, presentations, and other content as possible into a form and place which is accessible in the long term, free of charge. +# Be opionated + +One size does never fit all, so we need to deliberately choose what we want to +achieve and what not. We can not, and do not try to, fill every single niche. + # Play well with others Prometheus is a project of convinced and passionate individuals. As we do not @@ -58,7 +58,7 @@ meet arbitrary business requirements, we can focus on getting things right. This also means that we are free to suggest other implementations and projects if they are a better fit for a particular use-case. -# Be inclusive +# Be inclusive - TODO, move elsewhere We strongly believe that technology should be accessible to all. As such, we will always strive to be welcoming to everyone.