-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Image recognition, external URLs, content references (#44)
This introduces several features for the content creation dialog. - If the component itself or siblings contain references to other pages / content fragments / experience fragments / images, up to 5 of these are offered as sources in the source selection menu. If selected, their text content is retrieved into the source text. If an image is referenced, it is displayed instead as source content. - There is an additional selector "external URL" that makes an URL field appear. The text content of the URL will be used as source field. - If an image is selected from the source menu, it is possible to use the (beta) vision functionality of ChatGPT to e.g. generate a textual description.
- Loading branch information
Showing
51 changed files
with
1,792 additions
and
143 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # To get started with Dependabot version updates, you'll need to specify which | ||
| # package ecosystems to update and where the package manifests are located. | ||
| # Please see the documentation for all configuration options: | ||
| # https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates | ||
|
|
||
| version: 2 | ||
| updates: | ||
| - package-ecosystem: "maven" # See documentation for possible values | ||
| directory: "/" # Location of package manifests | ||
| schedule: | ||
| interval: "monthly" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,3 +33,4 @@ target | |
| .cgptdevbench/llmsearch.db | ||
| .linklint | ||
| .lycheecache | ||
| build.log | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
104 changes: 104 additions & 0 deletions
104
aem/core/src/main/java/com/composum/ai/aem/core/impl/ContentCreationSelectorsServlet.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| package com.composum.ai.aem.core.impl; | ||
|
|
||
| import java.io.IOException; | ||
| import java.io.InputStream; | ||
| import java.io.InputStreamReader; | ||
| import java.io.Reader; | ||
| import java.nio.charset.StandardCharsets; | ||
| import java.util.HashMap; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
| import java.util.stream.Collectors; | ||
|
|
||
| import javax.annotation.Nonnull; | ||
| import javax.servlet.Servlet; | ||
| import javax.servlet.ServletException; | ||
|
|
||
| import org.apache.sling.api.SlingHttpServletRequest; | ||
| import org.apache.sling.api.SlingHttpServletResponse; | ||
| import org.apache.sling.api.resource.Resource; | ||
| import org.apache.sling.api.resource.ResourceMetadata; | ||
| import org.apache.sling.api.resource.ValueMap; | ||
| import org.apache.sling.api.servlets.SlingSafeMethodsServlet; | ||
| import org.apache.sling.api.wrappers.ValueMapDecorator; | ||
| import org.osgi.framework.Constants; | ||
| import org.osgi.service.component.annotations.Component; | ||
| import org.osgi.service.component.annotations.Reference; | ||
|
|
||
| import com.adobe.granite.ui.components.ds.DataSource; | ||
| import com.adobe.granite.ui.components.ds.SimpleDataSource; | ||
| import com.adobe.granite.ui.components.ds.ValueMapResource; | ||
| import com.composum.ai.backend.slingbase.ApproximateMarkdownService; | ||
| import com.google.gson.Gson; | ||
|
|
||
| /** | ||
| * Servlet that reads the content selectors from a JSON file, adds links in the content and provides that to the dialog. | ||
| */ | ||
| @Component(service = Servlet.class, | ||
| property = { | ||
| Constants.SERVICE_DESCRIPTION + "=Composum Pages Content Creation Selectors Servlet", | ||
| "sling.servlet.resourceTypes=composum-ai/servlets/contentcreationselectors", | ||
| }) | ||
| public class ContentCreationSelectorsServlet extends SlingSafeMethodsServlet { | ||
|
|
||
| private final Gson gson = new Gson(); | ||
|
|
||
| /** | ||
| * JCR path to a JSON with the basic content selectors supported by the dialog. | ||
| */ | ||
| public static final String PATH_CONTENTSELECTORS = "/conf/composum-ai/settings/dialogs/contentcreation/contentselectors.json"; | ||
|
|
||
| @Reference | ||
| private ApproximateMarkdownService approximateMarkdownService; | ||
|
|
||
| @Override | ||
| protected void doGet(@Nonnull SlingHttpServletRequest request, @Nonnull SlingHttpServletResponse response) throws ServletException, IOException { | ||
| Map<String, String> contentSelectors = readPredefinedContentSelectors(request); | ||
| String path = request.getParameter("path"); | ||
| Resource resource = request.getResourceResolver().getResource(path); | ||
| if (resource != null) { | ||
| addContentPaths(resource, contentSelectors); | ||
| } | ||
| DataSource dataSource = transformToDatasource(request, contentSelectors); | ||
| request.setAttribute(DataSource.class.getName(), dataSource); | ||
| } | ||
|
|
||
| /** | ||
| * We look for content paths in the component and it's parent. That seems more appropriate than the component itself | ||
| * in AEM - often interesting links are contained one level up, e.g. for text fields in teasers. | ||
| */ | ||
| protected void addContentPaths(Resource resource, Map<String, String> contentSelectors) { | ||
| if (resource.getPath().contains("/jcr:content/")) { | ||
| resource = resource.getParent(); | ||
| } | ||
| List<ApproximateMarkdownService.Link> componentLinks = approximateMarkdownService.getComponentLinks(resource); | ||
| for (ApproximateMarkdownService.Link link : componentLinks) { | ||
| contentSelectors.put(link.getPath(), link.getTitle() + " (" + link.getPath() + ")"); | ||
| } | ||
| } | ||
|
|
||
| protected Map<String, String> readPredefinedContentSelectors(SlingHttpServletRequest request) throws IOException { | ||
| Resource resource = request.getResourceResolver().getResource(PATH_CONTENTSELECTORS); | ||
| Map<String, String> contentSelectors; | ||
| try (InputStream in = resource.adaptTo(InputStream.class); | ||
| Reader reader = new InputStreamReader(in, StandardCharsets.UTF_8)) { | ||
| contentSelectors = gson.fromJson(reader, Map.class); | ||
| } | ||
| return contentSelectors; | ||
| } | ||
|
|
||
| protected static DataSource transformToDatasource(SlingHttpServletRequest request, Map<String, String> contentSelectors) { | ||
| List<Resource> resourceList = contentSelectors.entrySet().stream() | ||
| .map(entry -> { | ||
| Map<String, Object> values = new HashMap<>(); | ||
| values.put("value", entry.getKey()); | ||
| values.put("text", entry.getValue()); | ||
| ValueMap valueMap = new ValueMapDecorator(values); | ||
| return new ValueMapResource(request.getResourceResolver(), new ResourceMetadata(), "nt:unstructured", valueMap); | ||
| }) | ||
| .collect(Collectors.toList()); | ||
| DataSource dataSource = new SimpleDataSource(resourceList.iterator()); | ||
| return dataSource; | ||
| } | ||
|
|
||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 changes: 9 additions & 0 deletions
9
.../content/jcr_root/conf/composum-ai/settings/dialogs/contentcreation/contentselectors.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "widget": "The text field you were editing", | ||
| "component": "The component you were editing, including subcomponents", | ||
| "page": "Current page text", | ||
| "lastoutput": "Current suggestion shown in this dialog (for iterative improvement)", | ||
| "url": "Text content of an external URL", | ||
| "empty": "No additional content", | ||
| "-": "Manually entered source content" | ||
| } |
25 changes: 0 additions & 25 deletions
25
.../jcr_root/conf/composum-ai/settings/dialogs/contentcreation/contentselectors/.content.xml
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.