Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Job docs to include info about enabling pod-to-pod communication within a job using pod hostnames #37771

Merged
merged 39 commits into from
Nov 30, 2022
Merged
Changes from 6 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7a17b9f
Update Job docs to include info about using a headless service to ena…
danielvegamyhre Nov 7, 2022
79be50e
Change section title
danielvegamyhre Nov 7, 2022
0cfbc3b
fix phrasing
danielvegamyhre Nov 8, 2022
477fd2d
update yaml example
danielvegamyhre Nov 8, 2022
6c88a52
update label selector
danielvegamyhre Nov 16, 2022
c6e7857
more specific phrasing
danielvegamyhre Nov 16, 2022
00abb8b
address comments and add new example
danielvegamyhre Nov 20, 2022
62cba84
add note about pod dns policies
danielvegamyhre Nov 20, 2022
a48dba8
minor fixes
danielvegamyhre Nov 20, 2022
003349e
add link to job patterns
danielvegamyhre Nov 20, 2022
7c0a4a0
Update content/en/docs/tasks/job/intra-job-pod-networking-using-pod-h…
danielvegamyhre Nov 21, 2022
b987524
Update content/en/docs/tasks/job/intra-job-pod-networking-using-pod-h…
danielvegamyhre Nov 21, 2022
10458e7
Update content/en/docs/tasks/job/intra-job-pod-networking-using-pod-h…
danielvegamyhre Nov 21, 2022
0973696
Update content/en/docs/tasks/job/intra-job-pod-networking-using-pod-h…
danielvegamyhre Nov 21, 2022
6741374
Update content/en/docs/concepts/workloads/controllers/job.md
danielvegamyhre Nov 21, 2022
b77d098
address comments
danielvegamyhre Nov 21, 2022
bf4272c
clarify sentence
danielvegamyhre Nov 21, 2022
1c001eb
move minikube note to prereqs
danielvegamyhre Nov 21, 2022
ea4b322
address comments
danielvegamyhre Nov 21, 2022
a23d3ab
captitalize all instances of Job
danielvegamyhre Nov 21, 2022
fb02aa3
move minikube notes to bottom of prereqs
danielvegamyhre Nov 21, 2022
e26da91
address comments
danielvegamyhre Nov 21, 2022
59879c0
update example
danielvegamyhre Nov 21, 2022
443e5e8
fix typo
danielvegamyhre Nov 21, 2022
a97339f
update phrasing
danielvegamyhre Nov 21, 2022
2863800
link to this from the completion modes section of the job docs
danielvegamyhre Nov 21, 2022
38114e5
address phrasing comments
danielvegamyhre Nov 22, 2022
34acaeb
add newlines to break up block of text
danielvegamyhre Nov 22, 2022
340520f
update phrasing
danielvegamyhre Nov 22, 2022
7575e90
update phrasing
danielvegamyhre Nov 22, 2022
52349df
Update content/en/docs/concepts/workloads/controllers/job.md
danielvegamyhre Nov 30, 2022
fbc586d
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
5a8b396
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
4d5b1b1
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
67950c4
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
ae413dd
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
6c2f31c
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
60a65e7
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
1bd542e
Update content/en/docs/tasks/job/job-with-pod-to-pod-communication.md
danielvegamyhre Nov 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions content/en/docs/concepts/workloads/controllers/job.md
Original file line number Diff line number Diff line change
Expand Up @@ -818,6 +818,74 @@ controller is tracking a Job using Pod finalizers by checking if the Job has the
annotation `batch.kubernetes.io/job-tracking`. You should **not** manually add
or remove this annotation from Jobs.

### Pod Communication Within a Job Using Pod Hostnames
danielvegamyhre marked this conversation as resolved.
Show resolved Hide resolved
Some jobs may benefit from using pod hostnames for pod networking rather than pod IPs.
danielvegamyhre marked this conversation as resolved.
Show resolved Hide resolved
For example, IndexedJobs automatically set the pod hostname to be in the format of
danielvegamyhre marked this conversation as resolved.
Show resolved Hide resolved
`${jobName}-${completionIndex}`, which can be used to deterministically determine
pod hostnames and enable pod networking *without* needing to create a client connection to
the Kubernetes control plane to obtain pod hostnames/IPs via API requests. This can be useful
for use cases where pod networking is required but we don't want to depend on a network
connection with the Kubernetes API server.


To enable pod-to-pod communication using pod hostnames, you must do the following:
1. Set up a [headless service](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services)
with a valid label selector for the pods created by your job. This will trigger `kube-dns` to cache the hostnames of the pods running your job. One easy way to do this is to use the `job-name: <your-job-name>` selector.
danielvegamyhre marked this conversation as resolved.
Show resolved Hide resolved
2. Update the template spec in your job with the following: `subdomain: <headless-svc-name>`
where `<headless-svc-name>` must match the name of your headless service
exactly.

Example:

```yaml

apiVersion: v1
kind: Service
metadata:
name: headless-svc
spec:
ports:
- port: 80
name: web
clusterIP: None # clusterIP must be None to create a headless service
selector:
app: nginx # must match job template spec label
---
apiVersion: batch/v1
kind: Job
metadata:
name: web
spec:
completions: 3
parallelism: 3
completionMode: Indexed
template:
metadata:
labels:
app: nginx
spec:
subdomain: headless-svc # has to match headless service name
restartPolicy: Never
containers:
- name: nginx
image: registry.k8s.io/nginx-slim:0.8
danielvegamyhre marked this conversation as resolved.
Show resolved Hide resolved
ports:
- containerPort: 80
name: web
```

After applying the example above, other pods can reach eachother over
the network using: `<pod-hostname>.<headless-service-name>`.
danielvegamyhre marked this conversation as resolved.
Show resolved Hide resolved

Example: pinging pod `web-1` from pod `web-0` using pod hostname:

```
root@web-0:/# ping web-1.headless-svc
PING web-1.headless-svc.default.svc.cluster.local (10.88.3.25) 56(84) bytes of data.
64 bytes from web-1.headless-svc.default.svc.cluster.local (10.88.3.25): icmp_seq=1 ttl=62 time=1.01 ms
64 bytes from web-1.headless-svc.default.svc.cluster.local (10.88.3.25): icmp_seq=2 ttl=62 time=0.248 ms
```

## Alternatives

### Bare Pods
Expand Down