Create guide for Machine Learning Engine operators

[mik-laj]

Description

Hello,

A guide that describes how to use Machine Learning Engine service operators would be useful.

We have an example DAG for this service, so the guide should not be a big challenge.

If anyone is interested in this task, I am willing to provide all the necessary tips and information.

Other guides are available:
https://airflow.readthedocs.io/en/latest/howto/operator/index.html

All example DAGs for GCP are available in the example_dags directory.

Best regards,

Kamil

Use case / motivation

N/A

Related Issues

Missing guide for GCP: #8209

[U-Ozdemir]

Hello, I'm very new to contribute to open source projects and to Airflow, but I would like to contribute to learn some stuff. I do have a background in Data Science and im shifting more to data engineering now.

I would like more information about it and also where can I see this example DAG for ML service?

Best regards,
Ozdemir

[mik-laj]

Hello,

I am very happy that you want to work on it. I prepared a guide that describes how to write new guides.
https://docs.google.com/document/d/1RLQiuADIX00uGh0Llfcvv8WyCcayD6Cwkx4MNnHvDOc/edit#heading=h.wchyosj84oht
This document is WIP. If you have questions, you can ask in the document comments or here.
All Examples for GCP are available in the example_dags directory. Example for ML Engine is also available.

Best regards,
Kamil

[ansh-lehri]

Hello, I am not able to understand what you are asking for. If you can just explain it more, it will be very useful.
Best regards,
Ansh

[alimgafar]

I'm new to Airflow and this project. I would like to offer my help to review and edit materials for the guide as this is a role I've had professionally in my past. Let me know if you can use such services.

[mik-laj]

@ansh08lehri Hello,

I would like an example that describes how to use the ML Engine operators. We have similar guides for other services, but we don't have a guide for MLEngine.
Other guides for GCP:
https://airflow.readthedocs.io/en/latest/howto/operator/gcp/index.html
My favourite: https://airflow.readthedocs.io/en/latest/howto/operator/gcp/natural_language.html

Best regards,

@alimgafar I am very interested in such services. We have a lot of guides, but it would be useful to review them and introduce improvements. I count on your suggestions.
If you want to start, I invite you to read the contribution guide.
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst
All docs for GCP are available here https://github.com/apache/airflow/tree/master/docs/howto/operator/gcp
If you have questions, you can also write on Apache Sllack - Kamil Breguła
https://apache-airflow-slack.herokuapp.com/
https://app.slack.com/team/UEGUANE59

[U-Ozdemir]

Hello I have a question.

1. Is there some kind of deadline in which the guide needs to be finished?
2. Also how do I submit the things that I have already got?

Best regards

[mik-laj]

@U-Ozdemir We try to use a logical clock instead of a real clock in the community. The order of operations is more important than the time the task was completed. Some tasks need to be completed earlier because other users may need to benefit more from some change, but there are no specific time limits set by the community rule.

If you want to share changes with the community then you must create PR.
https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request-from-a-fork
You should also create a commit beforehand. It describes your change
https://help.github.com/en/github/committing-changes-to-your-project

[alimgafar]

Kamil, Sorry about the delay responding. This email fell into Bulk messages. I’ll look over the information you’ve provided. Regards, Alim

…

On Apr 21, 2020, at 10:17 PM, Kamil Breguła ***@***.***> wrote: @ansh08lehri Hello, I would like an example that describes how to use the ML Engine operators. We have similar guides for other services, but we don't have a guide for MLEngine. Other guides for GCP: https://airflow.readthedocs.io/en/latest/howto/operator/gcp/index.html My favourite: https://airflow.readthedocs.io/en/latest/howto/operator/gcp/natural_language.html Best regards, @alimgafar I am very interested in such services. We have a lot of guides, but it would be useful to review them and introduce improvements. I count on your suggestions. If you want to start, I invite you to read the contribution guide. https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst All docs for GCP are available here https://github.com/apache/airflow/tree/master/docs/howto/operator/gcp If you have questions, you can also write on Apache Sllack - Kamil Breguła https://apache-airflow-slack.herokuapp.com/ https://app.slack.com/team/UEGUANE59 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[U-Ozdemir]

Hello, I have submitted a part of the guide. It can be used by others here working on it aswell. Any feedback is always welcome.

Best regards,
Ozdemir

[vuppalli]

@mik-laj @U-Ozdemir Hello! I am currently interning at Google this summer and this issue is very relevant to the work I am doing. If you are not working on this issue anymore, I would love to take over. When I am finished, I can create a new PR and give you some credit as well!

[mik-laj]

@vuppalli I've assigned you to this ticket. If you have questions, go ahead and ask.

I invite you to read the contribution guide.
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst

[U-Ozdemir]

@vuppalli Hello friend. Sorry I did not reply earlier, I would be very happy if you can take over. Currently I am busy with another project which has priority. So by all means use it and I hope it will help you a bit further.