Recursive search not working properly

[CommunyAS]

It seems that the recursive search function is not working properly.

When setting the search folder to a folder two (or more) levels higher in the hierarchy (i.e. to main_folder in a hierarchy main_folder > sub_folder > literature_notes), the plugin does not find any notes (although existing).

When setting the search folder to a folder one level in the hierarchy (i.e. sub_folder in a hierarchy as above) or the folder containing MD notes (i.e. literature_notes) is works.

Versions:

• MacOS 14.7 Sonoma
• Zotero 7.0.8
• MarkDB-Connect 0.1.3

[daeh]

Please report your versions (OS, Zotero, MDBC)

Upload your debugging logs (Tools > MarkDB-Connect Troubleshooting > Save Full Debugging Logs)

If you want to troubleshoot on your end, you can run this from Tools > Developer > Run JavaScript to see what files MDBC is finding

const listDirContents = async (dirpath) => {
  const items = []
  await Zotero.File.iterateDirectory(dirpath, (item) => {
    if (!item.name.startsWith('.')) {
      items.push(item)
    }
  })
  return items
}

const listFilesRecursively = async function* (dirpath) {
  // Does not follow symbolic links //
  const entries = await listDirContents(dirpath)
  for (const entry of entries) {
    const zfile = Zotero.File.pathToFile(entry.path)
    if (zfile.isReadable() && !zfile.isHidden() && !zfile.isSpecial() && !zfile.isSymlink()) {
      if (zfile.isDirectory()) {
        yield* listFilesRecursively(entry.path)
      } else if (zfile.isFile()) {
        yield entry
      }
    }
  }
}

const files = []
for await (const file of listFilesRecursively("YOUR-LOCAL-PATH-TO/main_folder")) {
  files.push(file)
}

return files

[CommunyAS]

Updated my first post with version numbers. Here is the log file:
MarkDBConnect-logs.json

[daeh]

You're getting an error

  "getFilesRecursively": {
   "msg": "ERROR: NotFoundError: File not found",
  },

I haven'r run into this before, but we can try to figure out which file is causing the issue. This code should print a log that tells you which file is terminating the process. You can paste the output here or just let me know what you learn about why the error is being evoked.

Tools > Developer > Run JavaScript

const log = []
const files = []

const listDirContents = async (dirpath) => {
    const items = []
    await Zotero.File.iterateDirectory(dirpath, (item) => {
        try {
            if (!item.name.startsWith('.')) {
                // log.push({ item , msg: "listDirContents, yielding item" })
                items.push(item)
            }
        } catch (err) {
            log.push({ path: item.path, error: err, msg: "ERROR (listDirContents)" })
        }
    })
    return items
}

const listFilesRecursively = async function*(dirpath) {
    // log.push({ dirpath , msg: "starting listFilesRecursively()" })
    const entries = await listDirContents(dirpath)
    // log.push({ dirpath: dirpath, nentries: entries.length , msg: "nentries" })
    for (const entry of entries) {
        try {
            // log.push({ path: entry.path , msg: "for entry" })
            const zfile = Zotero.File.pathToFile(entry.path)
            if (zfile.isReadable() && !zfile.isHidden() && !zfile.isSpecial() && !zfile.isSymlink()) {
                if (zfile.isDirectory()) {
                    // log.push({ path: entry.path , msg: "yielding entry dir" })
                    yield* listFilesRecursively(entry.path)
                } else if (zfile.isFile()) {
                    // log.push({ path: entry.path , msg: "yielding entry file" })
                    yield entry
                }
            }
        } catch (err) {
            log.push({ dirpath: dirpath.path, entrypath: entry.path, error: err, msg: "ERROR (listFilesRecursively)" })
        }
    }
}

async function getFilesRecursively(dirpath) {
    log.push({ dirpath, msg: "dirpath" })
    const basedirObj = Zotero.File.pathToFile(dirpath)
    log.push({ path: basedirObj.path, msg: "basedirObj" })

    basedirObj.normalize()
    log.push({ pathnormed: basedirObj.path, msg: "basedirObj normalized" })
    log.push({
        data: {
            path: basedirObj.path,
            exists: basedirObj.exists(),
            isdir: basedirObj.isDirectory(),
            readable: basedirObj.isReadable(),
            hidden: basedirObj.isHidden(),
            special: basedirObj.isSpecial(),
            symlink: basedirObj.isSymlink(),
            // directoryEntries: basedirObj.directoryEntries,
            permissions: basedirObj.permissions,
        },
        msg: "basedirObj info"
    })

    for await (const file of listFilesRecursively(basedirObj.path)) {
        files.push(file.path)
    }

    return files
}

try {
    const filelist = await getFilesRecursively("/Users/aspatzier/LAD_data")
} catch (err) {
    log.push({ files: files, error: err, msg: "ERROR (getFilesRecursively)" })
}

return log

[daeh]

You can also try the prerelease https://github.com/daeh/zotero-markdb-connect/releases/tag/v0.1.4-beta.1

I'm shooting in the dark a little bit, but it's possible some of these changes will resolve whatever is causing issues on your end. Let me know either way.

[CommunyAS]

I run the script and the log returns one specific MSG file (MS Outlook mail) to cause the error. Since I have about 80 such file stored in the folder, I don't know if this is only the first file found by the script or if it is the one and only file causing problems.

I also tried the beta version - the problems seems to be fixed, all md notes contained with the higher hierarchy folder are found.

[daeh]

I merged the updates to the v0.1.4 release https://github.com/daeh/zotero-markdb-connect/releases/tag/v0.1.4

Thanks for noticing the issue. Please report back if you notice anything else.