forked from kijiproject/kiji-scoring
-
Notifications
You must be signed in to change notification settings - Fork 0
/
RELEASE_NOTES.txt
138 lines (129 loc) · 9.81 KB
/
RELEASE_NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
Kiji Scoring Release Notes
Version 0.8.0
* SCORE-120. FreshKijiTableReader now supports providing an ExecutorService to use as the
thread pool backing the reader. If unspecified, the reader will use the singleton service
defined in FreshenerThreadPool.
Version 0.7.0
* SCORE-128. InternalFreshKijiTableReader now uses a reader pool to read data to check for
freshness and score in order to guarantee thread safety. The pool size defaults to the same
as the FreshenerThreadPool size.
* SCORE-118. InternalFreshKijiTableReader now includes DEBUG level log messages for important
events. Enable these messages through your slf4j configuration.
* SCORE-117. ScoreFunction.score() now returns a TimestampedValue which optionally specifies the
timestamp at which the value should be written to Kiji.
* SCORE-113. KijiScoring now includes a KijiProducer implementation called
org.kiji.scoring.lib.produce.ScoreFunctionProducer which can run a ScoreFunction in batch.
ScoreFunctions to be run in batch must meet certain requirements which are detailed in
ScoreFunctionProducer's JavaDoc.
* SCORE-112. FreshKijiTableReader.get(EntityId, KijiDataRequest, long) has been replaced by
get(EntityId, KijiDataRequest, org.kiji.scoring.FreshKijiTableReader.FreshRequestOptions)
which allows for the specification of an overriding timestamp and string-string parameters
which will override parameters available to the KijiFreshnessPolicy and ScoreFunction for
this request only.
* SCORE-111. FreshKijiTableReaderBuilder has been moved into the FreshKijiTableReader interface
and is now FreshKijiTableReader.Builder. All other FreshKijiTableReaderBuilder APIs remain
the same.
* SCORE-97. KijiScoring has undergone significant API changes. The KijiFreshnessPolicy SPI has
been redesigned and KijiProducer has been replaced by a new SPI called ScoreFunction. These
SPIs are designed specifically to support the use-case of real time scoring in a multi
threaded environment. To support the new SPIs KijiFreshnessPolicyRecord has been replaced
with KijiFreshenerRecord and KijiFreshnessManager has undergone a complete API revamp to
support the new record type. FreshTool was also changed to reflect the new records and SPIs.
FreshKijiTableBuiler has also seen several changes to support the new SPIs. Details of these
new classes can be found in the JavaDoc for each.
Version 0.6.0
* SCORE-40. FreshKijiTableReader now supports a `getStatistics()` method which returns
statistics about Fresheners run by the reader. Statistics gathering and logging can be
configured via `FreshKijiTableReaderBuilder.withStatisticsGathering()`.
* SCORE-92. FreshTool now supports registering and retrieving the new optional record fields
`parameters` and `reinitializeProducer`. e.g.
kiji fresh \
kiji://.env/default/table/family:qualifier \
--do=register \
--policy-class=my.Policy \
--policy-state="" \
--producer-class=my.Producer \
--parameters={"key":"value"} \
--reinitialize-producer=true
* SCORE-83. InternalFreshKijiTableReader now respects KijiFreshnessPolicyRecord's `parameters`
and `reinitializeProducer` fields. KijiFreshnessPolicy objects may access and override
`parameters` via PolicyContext.getParameters() and setParameter(String, String). These will
be written into the Configuration used by the KijiProducer and are available to the
KijiProducer objects via KijiProducer.getConf(). `reinitializeProducer` will instruct the
FreshKijiTableReader to intialize a new KijiProducer object for each request and configure
the new producer to include values found in the possibly modified version of `parameters`
returned from PolicyContext.getParameters(). A freshness policy may override the specified
reinitialization behavior by calling PolicyContext.reinitializeProducer(boolean) during a
call to KijiFreshnessPolicy.isFresh(KijiRowData, PolicyContext).
* SCORE-85. KijiFreshnessPolicyRecord now support two optional fields, paramters which is a
string-string mapping of configuration parameters which are made available to the policy
via PolicyContext.getParameters() and the producer via KijiProducer.getConf(), and
`reinitializeProducer` which is a boolean and instructs the FreshKijiTableReader to
initialize a new KijiProducer on each request in order to pass in new configuration
parameters safely.
* SCORE-81. KijiFreshnessManager now support writing multiple records with a single call. No
records will be written unless all records pass validation. Records to write together are
accepted in the same form that they are returned by retrievePolicies(tableName) to support
easy bulk modification of stored Freshener records.
This patch also fixes an error in attachment validation which was permitting records to be
attached to group type columns when another record was already attached.
* SCORE-82. FreshKijiTableReader now supports rereadPolicies(List<KijiColumnName>) which causes
the reader to reread policies from the metatable according to a new list of columns to
freshen. The new list of columns to freshen permanently supercedes the previous list.
* SCORE-75. InternalFreshKijiTableReader has been significantly refactored to improve
maintainability. FreshKijiTableReader.rereadPolicies() no longer takes a boolean parameter
to trigger preloading of KijiFreshnessPolicy and KijiProducer objects. Instead the reader
proactively instantiates all policy and producer objects at construction and on calls to
rereadPolicies. FreshKijiTableReader.preload(KijiDataRequest) has been removed as part of
this change. Users with many attached freshness policies may now use
FreshKijiTableReaderBuilder.withColumnsToFreshen(List<KijiColumnName>) to specify for which
columns the constructed reader should run freshening. This list of columns will be used by
calls to rereadPolicies() to determine which policies should be instantiated. This list of
columns is immutable and only policies attached to columns in this list will ever be
instantiated by this reader. Columns in this list may be fully qualified columns or
families. Including a family in this list will include all policies attached to qualified
columns within that family or the map family policy attached to the entire family.
Version 0.5.0
* SCORE-78. Prepares KijiScoring for BentoBox. Many unnecessary dependencies have been
removed. Missing newlines have been added in the FreshTool. README.md updated.
* SCORE-77, SCORE-74. KijiScoring now properly releases all Kiji resources.
FreshKijiTableReaders and KijiFreshnessManagers must be closed to release resources.
* SCORE-76. FreshTool's --help option no longer requires a meaningless --do flag.
Version 0.4.0
* SCORE-61. Resolve race conditions in InternalFreshKijiTableReader
* SCORE-69. Local FreshKijiTableReader now creates only one table connection for all write
operations.
* SCORE-54. Writing using the ProducerContext in setup and cleanup will now produce an
UnsupportedOperationException. If you wish to write during these methods, use a
KijiTableWriter instead.
* SCORE-32. NewerThan now exposes a getter getNewerThanTimeInMillis() to retrieve the configured
time before which data is considered stale.
* SCORE-52. FreshTool now accepts File paths for policy state using the --policy-state-file flag.
* SCORE-27. PolicyContext now supports a getStore(String) operation so that FreshnessPolicys can
access KeyValue stores in calls to isFresh().
Version 0.3.0
* SCORE-44. Producer output from freshened get() requests are now committed atomically based on
the 'allow partial freshness' rule specified during table creation. If partial freshness is
allowed each producer will commit all of its writes atomically after produce() returns. If
partial freshness is not allowed, all writes will be cached until all producers associated
with the same request have finished. Partial freshness requires a separate table connection
for each producer and does not guarantee that the output of one producer will not be used as
input to another. Given the high cost of table connections, partial freshening is not
recommended and is disabled by default.
Version 0.2.0
* SCORE-46. Freshness policies attached to qualified columns in map type families will now be
triggered by requests to the entire map type family.
* SCORE-49. Fixes ConcurrentModificationException in FreshKijiTableReader.rereadPolicies().
* SCORE-48. Updates pom.xml versions to latest released versions of dependencies.
* SCORE-47. KijiFreshnessManager and FreshTool now support post-attachment validation of
freshness policies.
* SCORE-45. FreshKijiTableReaderBuilder.get() is now FreshKijiTableReaderBuilder.create().
* SCORE-34. FreshKijiTableReader.reloadPolicies has been renamed rereadPolicies to reflect the
distinction from preload.
* SCORE-22. KijiFreshnessManager store policy methods now include a variety of safety checks.
* SCORE-41. FreshKijiTableReader now includes a preload method, which instantiates all objects
necessary for freshening a given data request without side effects.
* SCORE-39. FreshKijiTableReader now includes methods which override configured timeout durations.
* SCORE-13. FreshKijiTableReaderFactory has been replaced with FreshKijiTableReaderBuilder.
* SCORE-36. Makes the fresh CLI tool visible to the kiji tool launcher.
Version 0.1.0