diff --git a/README.md b/README.md index 7d962884c5c43..f494921d63ff9 100644 --- a/README.md +++ b/README.md @@ -29,26 +29,9 @@ $ yarn build This command generates static content into the `build` directory and can be served using any static contents hosting service. -### Deployment -Using SSH: +### Contributing -``` -$ USE_SSH=true yarn deploy -``` - -Not using SSH: - -``` -$ GIT_USER= yarn deploy -``` +If you see an issue in one of the examples, feel free to open a PR! Or file an issue reporting it. -If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch. - -### Continuous Integration - -Some common defaults for linting/formatting have been set for you. If you integrate your project with an open source Continuous Integration system (e.g. Travis CI, CircleCI), you may check for issues using the following command. - -``` -$ yarn ci -``` +Thanks for reading! \ No newline at end of file diff --git a/docs/evaluation/evaluator-implementations.mdx b/docs/evaluation/evaluator-implementations.mdx index 6d1f2ea472d16..1308a0d06813d 100644 --- a/docs/evaluation/evaluator-implementations.mdx +++ b/docs/evaluation/evaluator-implementations.mdx @@ -21,6 +21,7 @@ If you don't see an implementation that suits your needs, you can learn how to c :::note Most of these evaluators are useful but imperfect! We recommend against blind trust of any single automated metric and to always incorporate them as a part of a holistic testing and evaluation strategy. +Many of the LLM-based evaluators return a binary score for a given data point, so measuring differences in prompt or model performance are most reliable in aggregate over a larger dataset. ::: ## Correctness: QA evaluation diff --git a/docs/tracing/tracing-faq.mdx b/docs/tracing/tracing-faq.mdx index 5e40320efb82e..ec6e8af4f99e0 100644 --- a/docs/tracing/tracing-faq.mdx +++ b/docs/tracing/tracing-faq.mdx @@ -453,6 +453,16 @@ In python, we also provide an experimental `@traceable` decorator that automatic +### When logging with the SDK, which fields can I update when I patch? + +The following fields can be updated when patching a run: + +- end_time: `datetime.datetime` +- error: `str | None` +- outputs: `Dict | None` +- events: `list[dict] | None` + +Once an `end_time` is set on a run, it is marked as "closed" and can no longer be updated. This is the case if you include an end time in the initial run creation `post` request or if you do so in a later `patch` request. ### How do I search and filter runs?