F/search package #161

pranavkulkarni · 2019-04-30T19:21:29Z

Module to search for ArcGIS items and format them for display in ArcGIS Hub

Structure --

src
  ago
    helpers
      aggs
      filters
      format
      page
      sort
    search.ts // searches for items and returns raw ago results
    params.ts // defines ISearchParams interface to hub.js/search
    serialize.ts // serialize raw UI params into query parameters that V3 understands
    format-item-collection.ts // format raw ago results into hub v3 style datasets
    compute-items-facets.ts // compute aggs/facets based on ago results and also calculate aggs for custom facets

Sample of how this will be used in UI code --

const authentication = new arcgisRest.UserSession({
      portal: 'https://qaext.arcgis.com/sharing/rest',
      username: 'dcadminqa',
      password: 'dcadminqa1'
    })
    const portal = 'https://qaext.arcgis.com/sharing/rest'
    const params = {
      q: 'crime',
      sort: 'name',
      agg: { fields: 'tags,collection,owner,source,hasApi,downloadable' },
      start: 1,
      num: 10
    }
    authentication.getToken(portal)
    .then(token => {
      const headers = { authorization: token, portal }
      const serializedParams = arcgisHub.serialize(params)
      console.log(serializedParams)
      return fetch(`hub.arcgis.com/api/v3/search?${serializedParams}`, { headers })
    })
    .then(unifiedResultsPromise => {
      return unifiedResultsPromise.json()
    })
    .then(unifiedResults => {
      console.log(unifiedResults)
      // UPDATE UI model with unified results
    })

Sample of how hub indexer api v3 new search route will use hub.js --

   const search = require('@esri/hub-search')

  // 1. perform ago search with facets and formatting
  const agoResults = await search.agoSearch(req.query, req.headers.authorization, req.headers.portal)

  // 2. compute items facets
  const facets = await search.computeItemsFacets(agoResults.aggregations, req.query, req.headers.authorization, req.headers.portal)

  // 3. format item collection based on actual search results and computed facets
  const model = search.agoFormatItemCollection(agoResults, facets, req.query)

  return model

jgravois

hero's work @pranavkulkarni. this PR is really coming along.

besides my new comments, most everything in my first review still applies too.

jgravois · 2019-05-01T15:57:46Z

packages/search/package.json

@@ -0,0 +1,61 @@
+{
+  "name": "@esri/hub-search",
+  "version": "1.10.0",


1.11.1 (just so lerna will know to help us bump to v2.0.0)

jgravois · 2019-05-01T15:58:34Z

packages/search/package.json

+{
+  "name": "@esri/hub-search",
+  "version": "1.10.0",
+  "description": "Module to search for ArcGIS items and format them for display in ArcGIS Hub.",


it might be helpful to state explicitly that we're using this as a server-side module.

actually, we will be using this on front end too.

jgravois · 2019-05-01T16:09:54Z

test.html

@@ -0,0 +1,78 @@
+<!DOCTYPE html>


we don't want to check this one in.

It is there in .gitignore. Not sure why it has appeared 😕

i think you overrode the .gitignore at some point with an explicit git add test.html.

luckily you can git rm test.html too.

jgravois · 2019-05-01T16:10:25Z

packages/search/src/index.ts

@@ -0,0 +1,9 @@
+/* Copyright (c) 2018 Environmental Systems Research Institute, Inc.


jgravois · 2019-05-01T16:11:08Z

packages/search/src/ago/params.ts

@@ -0,0 +1,29 @@
+import { IRequestOptions } from "@esri/arcgis-rest-request";
+
+/* Copyright (c) 2017 Environmental Systems Research Institute, Inc.


2017 > 2019 (and move to the top of the file).

jgravois · 2019-05-01T16:14:26Z

packages/search/src/ago/params.ts

+/* Copyright (c) 2017 Environmental Systems Research Institute, Inc.
+* Apache-2.0 */
+
+export interface ISearchParams extends IRequestOptions {


ISearchOptions (see #152 (comment))

jgravois · 2019-05-01T16:16:16Z

packages/search/src/ago/helpers/filters/downloadable.ts

+
+const downloadableTypeKeywords = ["Data"];
+// eligible types are listed here: http://doc.arcgis.com/en/arcgis-online/reference/supported-items.htm
+const downloadableTypes = [


whether you keep this type in common or search, lets keep it DRY.

jgravois · 2019-05-01T16:18:01Z

packages/search/src/ago/helpers/aggs/downloadable.ts

+}
+
+// eligible types are listed here: http://doc.arcgis.com/en/arcgis-online/reference/supported-items.htm
+const downloadableTypes = [


given what we already started in @esri/hub-common, would it make more sense to export this type from there?

jgravois · 2019-05-01T16:23:29Z

packages/search/src/ago/compute-items-facets.ts

+ * require more complex and customized logic. Those go in the `customFacets` hash,
+ * where the name of the key is the name of the facet being computed and the custom function
+ * is implemented below.
+ */


this code comment isn't being picked up by TSDoc.

for an example of method documentation that's formatted for display in our documentation, see:

hub.js/packages/annotations/src/add.ts

Lines 29 to 50 in b336f32

/**

* ```js

* import { addAnnotations } from "@esri/hub-annotations";

* //

* addAnnotations({

* url: annotationsUrl + "/0",

* features: [{

* attributes: {

* target: "http://...", // required, explains what is being commented on

* description: "A grand idea!" // also required. this is the actual comment

* }

* }]

* })

* .then(response);

* ```

* Add an annotation to ArcGIS Hub. Uses authentication to derive authorship, appends a timestamp and sets a default status of "pending" to new comments by default.

* @param requestOptions - request options that may include authentication

* @returns A Promise that will resolve with the response from the service after attempting to add one or more new annotations.

*/

export function addAnnotations(

requestOptions: IAddAnnotationsRequestOptions

): Promise<IAddFeaturesResult> {

you can run npm run docs:serve to build the documentation locally and see it update live as you make changes to the raw source.

nice! I will fix that

jgravois · 2019-05-01T16:28:15Z

packages/search/src/ago/encode-ago-query.ts

+ * Construct a query object with filters, and queryParams sent by hub indexer
+ * @param queryObject any
+ */
+export function encodeAgoQuery(queryParams: any = {}) {


all exported functions show up in the documentation unless they are added to the explicit blacklist below.

i think it'd be appropriate to hide a lot of your shared utility functions, but i'll leave it up to you to decide exactly which.

hub.js/docs/build-typedoc.js

Line 138 in da48fbc

const blacklist = [];

const blacklist = [ "encodeAgoQuery", "etc" ];

jgravois · 2019-05-08T00:00:00Z

now that v2.0.0 has shipped, we'll land this from #152

pranavkulkarni added 13 commits April 30, 2019 10:52

search pkg basics

61e1eff

formatItemCollection added

3c9e8f1

sort field transform

d5ea50a

functions renamed, removed es2017

328d059

serialize func added

7e565ad

structural changes

73702f3

working serialize, searchItems

1b1973a

agg handled correctly

08225f2

item search categories (collections)

6fbddac

folder structuring, efficient aggregations

eff5f89

pagination using start and num

ebe40fb

Merge branch 'v2.0.0' into f/search-package

85b606c

merged with v2.0.0 branch, uses latest arcgis-rest-js features

2ee29ed

pranavkulkarni requested review from haoliangyu, jgravois and dmfenton April 30, 2019 19:21

added a few tests

27eda35

jgravois suggested changes May 1, 2019

View reviewed changes

jgravois requested a review from tomwayson May 1, 2019 16:34

pranavkulkarni added 6 commits May 1, 2019 17:10

more tests, improved comments, fixed collection bug

1d07ead

added tests for filters

07c6dc1

added more tests

f027802

100% test coverage achieved

f141d45

moved test.html

ea13e7b

downloadable types from common

c2ae3c1

jgravois closed this May 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

F/search package #161

F/search package #161

pranavkulkarni commented Apr 30, 2019

jgravois left a comment •

edited

Loading

jgravois May 1, 2019

jgravois May 1, 2019

pranavkulkarni May 1, 2019

jgravois May 1, 2019

pranavkulkarni May 1, 2019

jgravois May 1, 2019

jgravois May 1, 2019

jgravois May 1, 2019

jgravois May 1, 2019

jgravois May 1, 2019

jgravois May 1, 2019

jgravois May 1, 2019 •

edited

Loading

pranavkulkarni May 1, 2019

jgravois May 1, 2019

jgravois commented May 8, 2019

		@@ -0,0 +1,9 @@
		/* Copyright (c) 2018 Environmental Systems Research Institute, Inc.

		@@ -0,0 +1,29 @@
		import { IRequestOptions } from "@esri/arcgis-rest-request";

		/* Copyright (c) 2017 Environmental Systems Research Institute, Inc.

	/**
	* ```js
	* import { addAnnotations } from "@esri/hub-annotations";
	* //
	* addAnnotations({
	* url: annotationsUrl + "/0",
	* features: [{
	* attributes: {
	* target: "http://...", // required, explains what is being commented on
	* description: "A grand idea!" // also required. this is the actual comment
	* }
	* }]
	* })
	* .then(response);
	* ```
	* Add an annotation to ArcGIS Hub. Uses authentication to derive authorship, appends a timestamp and sets a default status of "pending" to new comments by default.
	* @param requestOptions - request options that may include authentication
	* @returns A Promise that will resolve with the response from the service after attempting to add one or more new annotations.
	*/
	export function addAnnotations(
	requestOptions: IAddAnnotationsRequestOptions
	): Promise<IAddFeaturesResult> {

F/search package #161

F/search package #161

Conversation

pranavkulkarni commented Apr 30, 2019

jgravois left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgravois May 1, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgravois commented May 8, 2019

jgravois left a comment •

edited

Loading

jgravois May 1, 2019 •

edited

Loading