- Bump dependencies.
- Bump the SDK version to
3.0.0
. - Remove the
meta
package. - Remove work-around for issue fixed by dart-lang/sdk#49188.
- Updated licence bearer.
- Small phrasing change in README.md.
- Compiled the example files into a single file
example.dart
.
- Added support for the
Crawl-delay
andHost
fields.
- Updated README.md.
- Added a method
.validate()
for validating files. - Renamed
parser.dart
torobots.dart
.
- Converted the
onlyApplicableTo
parameter inRobots.parse()
from aString
into aSet
to allow multiple user-agents to be specified at once. - Fixed the
onlyApplicableTo
parameter inRobots.parse()
not being taken into account.
- Additions:
- Added dependencies:
meta
for static analysis.
- Added developer dependencies:
test
for testing.
- Added support for the 'Sitemap' field.
- Added support for specifying:
- The precedent rule type for determining whether a certain user-agent can
or cannot access a certain path. (
PrecedentRuleType
) - The comparison strategy to use for comparing rule precedence.
(
PrecedenceStrategy
)
- The precedent rule type for determining whether a certain user-agent can
or cannot access a certain path. (
- Added tests.
- Added dependencies:
- Changes:
- Bumped the minimum SDK version to
2.17.0
for enhanced enum support.
- Bumped the minimum SDK version to
- Improvements:
- Made all structs
const
and marked them as@sealed
and@immutable
.
- Made all structs
- Deletions:
- Removed dependencies:
sprint
web_scraper
- Removed dependencies:
- Updated project description.
- Adapted code to lint rules.
- Improved documentation.
- Bumped year in the license.
- Updated package description.
- Updated dependency versions.
- Formatted files in accordance with
dartfmt
.
- Fixed the reading of the contents of
robots.txt
. - Fixed the parsing of rule fields to
Rule
s. - Added
example.dart
.
- Initial release.