Object finder implementation

[electrickery]

To find libraries containing a given object name, several models come to mind. Here a description with some pros and cons. Coupling with the Pd error message "... is not in scope. Maybe later.

• an object list per library uploaded to puredata.info
  • the maintainer should provide a list of objects(1),
  • the server can either:
    • concatenate and relay all found lists, combined with the library name and location. The client searches this list and presents matching libraries.
    • concatenate and search all found lists and returns the matching libraries. This option will cause less traffic and the smallest change in the client.
• a single list containing lines like "<objectName> <libraryName>".
  • simple server implementation;
    • search the list and provide names/urls
    • return the list. The client can search the list and then execute a traditional library name search
  • requires a central maintainer. Low workload expected, but this could change. When versions of libraries start to diverge, this model breaks down.

1. example: http://puredata.info/Members/fjkraan/software/tof/0.2.0/tof-v0.2.0-Objects.txt/view

[umlaeute]

i think/agree it would be easiest to move that logic to the server-side (only a single environment to consider; not having to worry about Tcl/Tk versions; not having to worry about Tcl).

i also think that the object-list should be maintained by the uploader, not by a central maintainer.

[umlaeute]

regarding format, i agree that the-simpler-the-better.

filename

• i totally like the idea that the filename is very similar to the deken-package filename
• however, to keep the name consistent with the deken-packages, i suggest using all lowercase for the reserved words.
• also I think that 'objects' is a bit ambiguous (compare with objectlist), but while writing this i realize that this is most likely just bike-shedding and it's very clear from the .txt extension that this file cannot contain real objects/externals/... but instead just a textual description.

contents

i think that @millerpuckette's idea of <library> <objectname> only makes sense when trying to resolve dependencies, not so much when declaring content of a package.
after all, the filename tof-v0.2.0 already contains the library name, and it contains additional information (the version).
to avoid inconsistencies, we should not allow to store the same information in multiple places.

otoh, it might be nice to have (optional) short descriptions (one-liners) of each object in the future:

crossfade~ crossfades 2 multi-channel sources
folderpanel query for a folder
getdollarzero get the $0 of a patch

caveats

it's a bit unclear to me how to attach a given objectlist to a deken package.

e.g. compare the simple (presumable consistent) case:

 http://puredata.info/Members/fjkraan/software/tof/0.2.0/tof-v0.2.0-objects.txt
 http://puredata.info/Members/fjkraan/software/tof/0.2.0/tof-v0.2.0-(Darwin-ppc-32)(Darwin-i386-32)(Darwin-x86_64-32)(Sources)-externals.tgz

to

 http://puredata.info/Members/foobar/software/tof-v0.2.0-objects.txt
 http://puredata.info/Members/fjkraan/software/tof/0.2.0/tof-v0.2.0-(Darwin-ppc-32)(Darwin-i386-32)(Darwin-x86_64-32)(Sources)-externals.tgz
 http://puredata.info/software/tof/0.2.0/tof-(Linux-i386-32)-externals.tgz
 http://puredata.info/Members/nastykid/tof-v0.2.0-objects.txt

[electrickery]

On 'filename':

objectlist.txt is ok with me, but if we start to add more info, object specific of library wide, it might become a misnominer. As only the server sees it, it could be partitioned into sections and section with library wide information added. Like author, maintainer, repository location. A properly generic name could then be 'tof-v0.2.0-metadata.txt'. Valid for all packages with the same prefix at the same puredata.info location.

On 'content':

For the original example I harvested the descriptions from the META sub-patch that Jonathan added to lots of pd-external help-patches. Adding those to the objects file should be no problem, but no interface is planned yet. The only change to the current deken GUI would be a selection between library and object search.

On 'caveats':

If the server does create the complete search repository, it could have the data organized as:

animate http://puredata.info/Members/fjkraan/software/tof/0.2.0/
argument http://puredata.info/Members/fjkraan/software/tof/0.2.0/
...
animate http://puredata.info/Members/nastykid/software/tof/0.2.0/
...

For each found objects file, the server remembers where it was found. The result send to the client is a list of library packages not unlike the current search result. The user still selects the preferred version of the library.

This assumes all library packages at a location and matching prefix contain all listed objects. Exceptions exist, but are not very common (AFAIK).

[umlaeute]

filename

metadata is nice, but then we should go for a structured file format (which in turn makes it easy to malformat)

consistency (aka caveats)

so the idea is basically to put trust into the fact that two files are living side-by-side?
probably makes sense, as

• this requires write-access to a given folder (where the -externals file also lives), something that probably requires enough determination to scare of the average script-kiddy.
• we already do this for the .gpg signature and the .sha256 hash.

[electrickery]

Maybe it is best to keep the score narrow for now, and add features when desired and worked out. I added *-objects.txt files for creb/0.9.2_darcs/, maxlib/1.5.6/, cyclone/0.2beta1/, sigpack/0.044/, freeverb~/1.2.2/ and tof/0.2.0/.

I took the liberty to allow an arbitrary number of spaces between object name and description in an attempt to make it somewhat more human readable.

[electrickery]

For an object search some modification is needed for the GUI plugin. So far I identified two possible approaches:

• add search type to the search field only, like "objects:decimate~". The results pane could just list the packages as it does for library search. Optionally the object description field might be printed. A bit geeky, but Google uses something similar.
• add some radio boxes selecting between "Library" and "Object" search. See the mockup.
  

[umlaeute]

yes.

i like both approaches (i actually thought about using tabs for the two search flavours, but radio-buttons is even easier).

i think we could also allow objects:foo in the libraries search (and vice versa).

[umlaeute]

after rethinking this, i think we can do it easier:

• leave the GUI interface as is
• if someone searches for a simple expression (e.g. net), it will return both libraries that match the term (e.g. iemnet & net (mrpeach)) and libraries containing matching objects (e.g. maxlib, as it contains netclient)
  • this should be good enough for most people
• when specifying the search-domain (e.g. library:net), the search will be performed only in that domain (e.g. only iemnet resp. net; but not maxlib)

[umlaeute]

i also think that specifying multiple search terms should OR the results.

common search knowledge has it, that when providing multiple values in one search-domain the results should be **OR**ed, whereas multiple search-domains should be **AND**ed. (libraries:iem* libraries:mrpeach objects:udp* would find any library matching iem* OR mrpeach, but only if they contained an object matching udp*).

while this makes total sense, it complicates things greatly if we allow unqualified search-terms (without the libraries: or objects: prefix) that apply to both.

[electrickery]

Combining library and object search results will make the results a bit harder to read. Not much a problem for the well informed, but these are not the whole target audience. So a simple unambigious mode looks like the best default mode. With complex expressions, the user should be prepared for complex results.

Combined results are complex because the library results have to be differentiated from the object results containing the library name and the expanded object name.

It would be great if we can find a mid-way between geek mode and a completely applelized interface.

[electrickery]

Mixing library and object results will add to the already existing confusion of result interpretation. Here is a proposal for displaying the two types. The first change is to display the platform relevant packages only. The other is to display only parts of the package name. For the object results, the library name is added between brackets. When the object description will be available, it could be added at the end. For functional different synonyns, this could be helpful.

Library result (mrpeach):
mrpeach v0.0.extended chr15m 2015-08-07 14:11:10

Object result (udp*):
udpsend (mrpeach v0.0.extended) chr15m 2015-08-07 14:11:10 "used in conjunction with packOSC - will send OSC over udp"

[umlaeute]

actually i was thinking to not have any special output for object output (this also means that for now i plan to ignore any additional information in the objects file; this could be changed in the future).

so searching for life2x would output exactly the same result as search for mrpeach (given that mrpeach is the only library that contains a life2x object).

i hope to get the server-side ready this week, so we could just try it out (as in: a search will start showing results from the object-search as well, for all users of the deken plugin)

[umlaeute]

try it out (the server is now running in a search libraries & objects mode)

[electrickery]

Currently downloading produces an error:

couldn't open socket: Name or service not known
couldn't open socket: Name or service not known
while executing
"http::geturl $URL -binary true -progress "::deken::download_progress" -channel $f"
(procedure "::deken::download_file" line 6)
invoked from within
"::deken::download_file $URL $fullpkgfile"
(procedure "::deken::clicked_link" line 37)
invoked from within
"::deken::clicked_link http://pdzope.iemnet:10080/pd/pd/Members/fjkraan/software/maxlib/1.5.6/maxlib-v1.5.6-(Linux-i686-32)(Sources)-externals.tgz maxl..."
(command bound to event)

[umlaeute]

yes, that's just my normal modus operandi :hudriwudri:
should work now...

[electrickery]

Yup, it works now, both libraries and objects. Will test some more.

[umlaeute]

i think this can be closed as "implemented"