Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Center deploy button #694

Closed
wants to merge 18 commits into from
Closed

[WIP] Center deploy button #694

wants to merge 18 commits into from

Conversation

ebamberg
Copy link
Contributor

@ebamberg ebamberg commented Feb 25, 2021

Description

@roywei @anfee1
here is the deploy-action button in Central I mentioned yesterday on Slack.
this change is not yet finished, but it creates functionality similar to the download=button that @anfee1 is about to implement.

image
The button-list is defined in
/central/src/main/webapp/components/ModelViewActions.jsx
it should be quite simple to add another download button to this actions like:

import SaveIcon from '@material-ui/icons/Save';
.
.
.
>

				<SpeedDialAction
					key={'Deploy'}
					icon={<ShareIcon />}
					tooltipTitle={'deploy'}
					onClick={deploy}
			
				/>
				**<SpeedDialAction
					key={'Download'}
					icon={<SaveIcon />}
					tooltipTitle={'download'}
					onClick={download}**
			
				/>
		</SpeedDial>

In the deploy-function declared in the same file you can see an example how to call the backend providing values from the current selected model:

const deploy = () => {
**axios.post("http://"+window.location.host+"/serving/models?**modelName="+props.modelName+"&url="+encodeURIComponent(props.modelUri))
.then(function(response) {
console.log(response.data.message);
alert(response.data.message)
})
.catch(function(error) {
console.log(error);
})
.then(function() {
// always executed
});
};

"axios"-framework is used to do the Rest-call to the backend and trigger a function in the backend.
when we receive an response from backend then the call in ".then(function()...) is called.
(see /central/src/main/java/ai/djl/serving/central/handler/ModelDeploymentHandler.java which handles requests for the url /serving/models/...)

The models are already loaded into central-UI. modelZoo.listModel() gets invoked when open the UI.
see the useFetch-function in /central/src/main/webapp/components/ModelNavigator.jsx .

The the website already holds information for the models in browser-memory.
when you select a model in to Tree then
/central/src/main/webapp/components/ModelView.jsx
is used to display the model details.
The component ModelView has a property called "model" which is exactly the model-info of this model.
you can access values from the model using "{props.model.version}" or "{props.model.files.parameters.uri}" and so on.

So I think we do not have to call listModel in the backend again to implement the download function.

We just have to trigger the backend-download using a axios-Rest-Call to trigger the download. Or if we don't want to use axios a href.location=.... would do the trick as well. maybe we don't even need the backend in this case cause the browser can directly download the model from this url. But imo going through the backend is the cleaner option when we are thinking about clouds, firewalls, permissions,proxies and so on which can all be handled by the backend-host.

Then we need a Netty-ChannelInbound-Handler in the backend to download the data and stream to bytes to the "Response"-object Chunk-by-Chunks (netty can process large files in chunks without storing the whole file in memory first !!)
We just have to set the http-Header of the response correctly so that the file is downloaded and not interpreted as html content by the browser.
This can be done by setting Content-Disposition to attachment
response.setHeader("Content-Disposition", "attachment; filename=" + YOUR_FILE_NAME);

examples for the Netty-ChannelInbound-Handler can be found in
package ai.djl.serving.central.handler
you have to register a new handler in
/central/src/main/java/ai/djl/serving/central/handler/HttpStaticFileServerInitializer.java.

I think /central/src/main/java/ai/djl/serving/central/handler/HttpStaticFileServerHandler.java
is a good example of a Netty-ChannelInbound-Handler for download cause this handler sends static-file-resource from the folder to the client and it's quite similar to downloading a file when a particular url is called.
in line 196 you also can see that the filecontent is sent chunk-wise to the client.
} else {
sendFileFuture =
ctx.writeAndFlush(
new HttpChunkedInput(new ChunkedFile(raf, 0, fileLength, 8192)),
ctx.newProgressivePromise());

// HttpChunkedInput will write the end marker (LastHttpContent) for us.
lastContentFuture = sendFileFuture;
}

this handler is mainly inspired by the FileServer example I had. This fileserver is doing mostly what you want to do. It downloads a files ! :-)
maybe this resources can be helpful:
https://netty.io/wiki/user-guide-for-4.x.html
https://github.com/netty/netty/tree/4.1/example/src/main/java/io/netty/example/file

this would be a bit simpler when using a framework like spring,, but netty was already used in the project and we can avoid dependencies to huge 3rd party frameworks this way.

I didn't try this solution, these are just my first thought how I would access this problem.
Just ping me if you have any questions.

sorry if this is to detailed :-)

Brief description of what this PR is about

  • If this change is a backward incompatible change, why must this change be made?
  • Interesting edge cases to note here

@anfee1 anfee1 mentioned this pull request Mar 2, 2021
@lanking520 lanking520 changed the title Center deploy button [WIP] Center deploy button Mar 2, 2021
@zachgk
Copy link
Contributor

zachgk commented Mar 18, 2021

I think this PR was mostly just here as a reference for #719. As that is now completed, I will close this one as well. If I am mistaken, please feel free to reopen

@zachgk zachgk closed this Mar 18, 2021
@ebamberg ebamberg deleted the center_deploy_button branch November 26, 2021 17:07
Lokiiiiii pushed a commit to Lokiiiiii/djl that referenced this pull request Oct 10, 2023
* add neuronx new feature for generation

* swap back
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants