Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need to define what it is to be "LCF-compliant" #258

Closed
franciscave opened this issue May 18, 2020 · 28 comments · Fixed by #279
Closed

Need to define what it is to be "LCF-compliant" #258

franciscave opened this issue May 18, 2020 · 28 comments · Fixed by #279
Assignees
Milestone

Comments

@franciscave
Copy link
Collaborator

This issue was raised at the LCF Technical Panel meeting on 14 May 2020. LCF implementers need to be able to give clear information on whether their implementations are LCF-compliant or not and in what kind of way they are compliant. It was agreed that a set of compliance criteria should be defined and a compliance test suite should be developed.

@franciscave
Copy link
Collaborator Author

Testable criteria for LCF v1.2.0 compliance

General criteria

"All web service implementations must identify the version of LCF that is implemented for each function"

Client: N/A

Server: Response header must include HTTP custom header field: lcf-version: 1.2.0.

Server requires terminal authentication

Client: Must be able to test whether the server requires terminal authentication, by issuing a request with no terminal authentication, then reacting correctly to an HTTP 401 response.

Server:

  1. If not using HTTP BASIC authentication, must be able to confirm to all client implementers, with documentation if appropriate, the selected method of terminal authentication.
  2. Must be able to respond correctly to a request not using the selected method of terminal authentication.

Server requires patron authentication

Client: Must be able to test whether the server requires terminal authentication, by issuing a request with no patron authentication, then reacting correctly to an HTTP 403 response.

Server:

  1. Must implement one of the two methods specified in LCF v1.2.0 REST Web Services Implementation, i.e. EITHER include a custom header field (lcf-patron-credential) in the request, OR include patron ID and password as query parameters in the request.
  2. Must be able to confirm to all client implementers the selected method of patron authentication.
  3. Must be able to respond correctly to a request not using the selected method of patron authentication.

Client may request a list of Patron access rights and privileges

Client: N/A

Server: Respond correctly to a request for a list of authorisations for a specified Patron (see Implementation Note 4 in the LCF v1.2.0 REST Web Services Implementation).

@franciscave
Copy link
Collaborator Author

Testable criteria relating to specific implementation profiles are to follow.

@franciscave
Copy link
Collaborator Author

franciscave commented May 21, 2020

Testable criteria for LCF v1.2.0 compliance (2)

Implementation Profile: P00 Core LMS Profile

This profile applies to any server that provides a communication front-end to an LMS. Such servers must implement all the functions in this profile to comply with the Core LMS Profile, i.e. they must be able to respond appropriately to all requests from clients. A client implementation is not required to implement all or any of the requests in this profile.

For assessment purposes, it would be useful if a server were configured to recognise a user who is authorised to perform all the functions that need to be tested. Whoever is carrying out the tests would need to satisfy themselves that the test environment is sufficiently close to a live environment that the results can be judged to be applicable to the live system.

A sensible response should therefore be obtainable for all the functions in the profile table for which there is a tick against an entity type. This can be translated into the following tests:

Function 02

As a minimum, all the following requests should elicit an appropriate response:

GET https://{server-host-name-or-IP-address}/lcf/1.0/authorisations
GET https://{server-host-name-or-IP-address}/lcf/1.0/manifestations
GET https://{server-host-name-or-IP-address}/lcf/1.0/items
GET https://{server-host-name-or-IP-address}/lcf/1.0/patrons
GET https://{server-host-name-or-IP-address}/lcf/1.0/locations
GET https://{server-host-name-or-IP-address}/lcf/1.0/loans
GET https://{server-host-name-or-IP-address}/lcf/1.0/reservations
GET https://{server-host-name-or-IP-address}/lcf/1.0/charges
GET https://{server-host-name-or-IP-address}/lcf/1.0/payments
GET https://{server-host-name-or-IP-address}/lcf/1.0/contacts
GET https://{server-host-name-or-IP-address}/lcf/1.0/authorities
GET https://{server-host-name-or-IP-address}/lcf/1.0/messages

The appropriate response in each case should be to list those entities to which the terminal user is authorised to have access.

Function 01

As Function 02 above, using one of the list of entity references provided in the response to each of the Function 02 requests above. The appropriate response should be to return the requested entity in the payload of the response, unless there is an exception condition.

Function 03

The following should elicit an appropriate response when accompanied by an XML payload containing a new record of the corresponding type:

POST https://{server-host-name-or-IP-address}/lcf/1.0/patrons
POST https://{server-host-name-or-IP-address}/lcf/1.0/reservations
POST https://{server-host-name-or-IP-address}/lcf/1.0/charges
POST https://{server-host-name-or-IP-address}/lcf/1.0/payments
POST https://{server-host-name-or-IP-address}/lcf/1.0/contacts

Function 04

The following should elicit an appropriate response when accompanied by an XML payload containing a modified record of the corresponding type:

PUT https://{server-host-name-or-IP-address}/lcf/1.0/items
PUT https://{server-host-name-or-IP-address}/lcf/1.0/patrons
PUT https://{server-host-name-or-IP-address}/lcf/1.0/contacts

The modified record would in each case be created by first retrieving a record of the corresponding type using Function 01 and making modifications to it.

Function 05

The following should elicit an appropriate response:

DELETE https://{server-host-name-or-IP-address}/lcf/1.0/patrons/{patron-id}
DELETE https://{server-host-name-or-IP-address}/lcf/1.0/reservations/{reservation-id}

The identifier of the record to be deleted would be obtained by first using Function 02 to obtain a list of records of the appropriate type.

@franciscave
Copy link
Collaborator Author

Testable criteria for LCF v1.2.0 compliance (3)

Implementation Profile: P01 Self Issue

In this profile there are five function groups labelled 'Basic', 'Security', 'Loans', 'Reservations' and 'Payments'. Compliance involves both clients and servers implementing some or all of these functional groups.

The first two functional groups are alternatives:

  • 'Basic' implies a minimal response to a check-out or check-in request, with no response payload;
  • 'Security' implies that, if there is any applicable item security information, the response must include a payload that provides this information.

The remaining function groups represent additional functionality that may be included in an implementation. This therefore suggests that there are a number of compliance levels:

For clients:

  • 'Basic' only: the client can issue check-out and check-in requests, and handle correct responses, confirm success or otherwise to the user, but ignore response payloads;
  • 'Security' only: the client can issue check-out and check-in requests, and handle correct responses, confirm success or otherwise, together with any item security information, to the user;
  • 'Basic' or 'Security' plus 'Payments' (no authorization required): on either check-out or check-in the client will report to the user any associated charges included in the server response; the client can subsequently issue a payment request with reference to the specific charges;
  • 'Basic' or 'Security' plus 'Payments' (authorization required): on either check-out or check-in the client will report to the user any associated charges included in the server response; the client will support a two-stage payment process, first to request initiation of the payment process and second to provide confirmation that the payment transaction has been completed;
  • 'Basic' or 'Security' plus 'Loans', 'Reservations' or both: in addition to check-out and check-in, the client enables the use to retrieve a list of loans (checked-out items) and/or a list of reservations.

For servers:

  • 'Basic' only: the server handles check-out and check-in requests correctly and provides a header response only (i.e. no payload);
  • 'Security' only: the server handles check-out and check-in requests correctly and provides a response with valid payload, including item security information;
  • 'Basic' or 'Security' plus 'Payments' (no authorization required): the server's response to a check-in or check-out response includes a charge reference when applicable; the server will then accept a payment request and respond correctly to indicate that payment has been accepted;
  • 'Basic' or 'Security' plus 'Payments' (authorization required): the server's response to a check-in or check-out response includes a charge reference when applicable; in response to an initial payment request, the server response with HTTP respond code 202; in response to a confirmation payment request, the server will depend upon whether the confirmation with transaction reference has been accepted;
  • 'Basic' or 'Security' plus 'Loans', 'Reservations' or both: in addition to check-out and check-in requests, the server responds appropriately to a request for a list of loans or reservations.

My assumption is that the point of this implementation profile is that both check-out and check-in are supported, as a minimum at the 'Basic' level, so we can discount implementation of other permutations of the function groups.

@franciscave
Copy link
Collaborator Author

A question concerning implementation profile P02 (patron information maintenance):

Should it be possible to create a new contact record? If a patron wishes to add a new additional contact phone number, as an alternative to the one already held on record, this would require the creation of a new contact record for that phone number.

If so, we would need to add creation of a new contact record to this profile.

@jeremylangley
Copy link

While some libraries services may allow for this, when looking at library services such as academic, school, special libraries - contact details may not come from the LMS, instead a third party IdP. Therefore while it can be supported, I do not believe that it should be a requirement.

@franciscave
Copy link
Collaborator Author

Understood.

@franciscave
Copy link
Collaborator Author

At the Technical Panel meeting on 21 July it was agreed that, to comply with implementation profile P01 Self Issue, security must be implemented. Furthermore, to implement payments, authorization must be supported, so the distinction between 'Payments' with and without authorisation disappears.

The minimal functional group for check-in and check-out is 'Basic', which implies that, if there is any applicable item security information, the response must include a payload that provides this information. This simplifies the options to the following:

For clients the options are now:

  • 'Basic' only: the client can issue check-out and check-in requests, and handle correct responses, confirm success or otherwise, together with any item security information, to the user;
  • 'Basic' plus 'Payments': on either check-out or check-in the client will report to the user any associated charges included in the server response; the client will support a two-stage payment process, first to request initiation of the payment process and second to provide confirmation that the payment transaction has been completed;
  • 'Basic' plus 'Loans' and 'Reservations': as 'Basic', plus the client enables the user to retrieve a list of a patron's loans (checked-out items) or reservations.
  • 'Basic' plus 'Payments', 'Loans' and 'Reservations': all the function groups are supported.

For servers the options are now:

  • 'Basic' only: the server handles check-out and check-in requests correctly and provides a response with valid payload, including item security information;
  • 'Basic' plus 'Payments': the server's response to a check-in or check-out response includes a charge reference when applicable; in response to an initial payment request, the server response with HTTP respond code 202; in response to a confirmation payment request, the server will depend upon whether the confirmation with transaction reference has been accepted;
  • 'Basic' plus 'Loans' and 'Reservations': as 'Basic', plus the server responds appropriately to a request for a list of a patron's loans or reservations.
  • 'Basic' plus 'Payments', 'Loans' and 'Reservations': all the function groups are supported.

@franciscave
Copy link
Collaborator Author

franciscave commented Aug 18, 2020

Client compliance with P01

P01 Basic

For a client to comply with P01 Basic, it must be able to issue each of the following requests in appropriate circumstances:

POST http://{server-host-name-or-IP-address}/lcf/1.0/loans
POST http://{server-host-name-or-IP-address}/lcf/1.0/loans?confirmation=Y

The request must be accompanied by a payload containing a valid Loan record as detailed in LCF Loan entity E05.

The client must be able to handle server responses, including both success and failure responses, and present the result to the user, including any item security information returned in the response payload.

P01 Basic plus Payments

For a client to comply with P01 Basic plus Payments, it must be able to issue the following request when the server response to a check-out request indicates that a payment is required:

POST http://{server-host-name-or-IP-address}/lcf/1.0/loans?charge-acknowledged=Y

A client must also be able to issue a payment request in two stages:

First stage

POST http://{server-host-name-or-IP-address}/lcf/1.0/payments

The request must be accompanied by a payload containing a valid Payment record as detailed in LCF Payment entity E08.

The client must be able to handle an HTTP 202 response from the server, with an authorisation reference in the response payload (see LCF function 13 Patron payment, R13D04 Authorisation reference).

Second stage

Having carried out the payment transaction (outside the scope of LCF), the client must be able to repeat the payment request:

POST http://{server-host-name-or-IP-address}/lcf/1.0/payments?confirmation=Y

The request must be accompanied by a payload containing a valid Payment record as detailed in LCF Payment entity E08, but this time including the authorisation reference and transaction reference in the request payload.

@franciscave
Copy link
Collaborator Author

I've noticed that the E08 Payment does not currently support the inclusion of an authorisation reference, i.e. a reference to an entity of type E13. I'll post a separate issue on this.

@franciscave
Copy link
Collaborator Author

See #269.

@franciscave
Copy link
Collaborator Author

Client compliance with P01 (continued)

P01 Basic plus Loans and Reservations

The same as P01 Basic, but in addition the client must be able to retrieve a patron's loans and reservations using a suitable combination of the following requests:

To retrieve a Patron record:

GET https://{server-host-name-or-IP-address}/lcf/1.0/patrons/nnnnnnnn

To retrieve a list of the Patron's Loans:

GET https://{server-host-name-or-IP-address}/lcf/1.0/patrons/nnnnnnnn/loans

To retrieve a list of the Patron's Reservations:

GET https://{server-host-name-or-IP-address}/lcf/1.0/patrons/nnnnnnnn/reservations

To retrieve a specific Loan:

GET https://{server-host-name-or-IP-address}/lcf/1.0/loans/mmmmmmmm

To retrieve a specific Reservation:

GET https://{server-host-name-or-IP-address}/lcf/1.0/reservations/pppppppp

P01 Basic plus Payments, Loans and Reservations

The client must support all the functionality of both P01 Basic plus Payments and P01 Basic plus Loans and Reservations.

@franciscave
Copy link
Collaborator Author

Client compliance with P01 (continued)

For P01 Basic plus Payments it may also be desirable (mandatory?) that the client be able to request lists of charges on a patron's account, as well as any payments already made relating to those charges:

GET http://{server-host-name-or-IP-address}/lcf/1.0/patrons/nnnnnnnn/charges
GET http://{server-host-name-or-IP-address}/lcf/1.0/charges/mmmmmmmm/payments
GET http://{server-host-name-or-IP-address}/lcf/1.0//payments/pppppppp

Being able to select charges for which payments have not yet been made might be desirable, but probably should not be made mandatory, as a requirement to be able to include selection criteria in requests might be a step too far at this stage.

@franciscave
Copy link
Collaborator Author

I now realise that I haven't yet considered the requirements of Check-in, only Check-out / renewal.

Server compliance with P01 (Check-out / renewal only)

P01 Basic

The server must be able to accept the following client requests and respond appropriately, processing the XML payload attached to the client request:

POST http://{server-host-name-or-IP-address}/lcf/1.0/loans
POST http://{server-host-name-or-IP-address}/lcf/1.0/loans?confirmation=Y

P01 Basic plus Payments

The server must be able to accept the following client request and respond appropriately, processing the XML payload attached to the client request:

POST http://{server-host-name-or-IP-address}/lcf/1.0/loans?charge-acknowledged=Y

First stage

In response to the following client request, the server must be able to interpret the XML payload attached to the client request, and, if the request is accepted (HTTP 202), including an authorisation reference in the required response payload (see the LCF REST Web Service Specification, function 11, for details):

POST http://{server-host-name-or-IP-address}/lcf/1.0/payments

Second stage

The server must be able to accept the following client request, processing the XML payload attached to the client request, checking that the client payload contains the necessary authorisation reference issued previously and a valid transaction reference, and responding appropriately:

POST http://{server-host-name-or-IP-address}/lcf/1.0/payments?confirmation=Y

@franciscave
Copy link
Collaborator Author

For details of the revised description of implementation profile P02, see #266.

@franciscave
Copy link
Collaborator Author

franciscave commented Aug 24, 2020

Client compliance with P01 for Check-in

P01 Basic

The client should be able to retrieve the loan that relates to the item(s) being checked in, e.g.

GET http://{server-host-name-or-IP-address}/lcf/1.0/items/nnnnnnnnnn/loans?status=01

The client should be able to retrieve the loan(s) that relate to the patron in question, e.g.

GET http://{server-host-name-or-IP-address}/lcf/1.0/patrons/xxxxxxxxxx/loans?status=01

The client should be able to retrieve the loan in question:

GET http://{server-host-name-or-IP-address}/lcf/1.0/loans/mmmmmmmmmm

The client should be able to update the loan in question:

PUT http://{server-host-name-or-IP-address}/lcf/1.0/loans/mmmmmmmmmm

The PUT request must be accompanied by the updated loan in the request payload.

Server compliance with P01 for Check-in

P01 Basic

The server must be able to respond correctly to all the client requests associated with check-in and must be able to process a PUT request to update the loan, that attempts to change its status from '01' (on loan to patron) to '08' (checked-in). An appropriate payload must be returned in response to a PUT request, including any security information relevant to the item being checked in.

@franciscave
Copy link
Collaborator Author

I don't believe that P01 should include cancellation of either check-out or check-in, as I believe it was decided that these require a staff override and should not be supported by self-issue.

@franciscave
Copy link
Collaborator Author

Assuming that the revised specifications of P00 and P02 proposed under issue #266 are adopted, here is a revised specification of P01 in the same format.

Possible consequent issue: Should it be possible within Profile P01.2 Reservations for a patron to cancel a reservation? Currently Profile P01 is unclear on this point. There is no specific function in LCF for cancelling a reservation, so it would involve implementation of function 04 Modify entity item, applied to entities of type E06 Reservation, specifically to set the Reservation Status (E06D11) to RVS03 'Reservation cancelled by patron'. We either need to specify a limited implementation of function 04, or define a new specific function for patron cancellation of a reservation.

P01 Self Issue (substantially revised in Issue 3)

Implementation of this profile involves implementation of one or more of the following four function groups:

P01.1: Basic self-issue, including check-out, renewal and check-in.
P01.2: Reservations, including making reservations
P01.3: Payments, including the ability of the terminal user to make payments and of the server to accept payments.

P01.1 Basic self-issue

In addition to the core functions specified in Profile P00, the following functions must be supported:

  • 11 Check-out/renewal
  • 12 Check-in

P01.2 Reservations

In addition to the core functions specified in Profile P00, the following function must also be supported:

  • 16 Reserve manifestation / item

P01.3 Payments

In addition to the core functions specified in Profile P00, the following function must also be supported:

  • 13 Patron payment

franciscave added a commit that referenced this issue Mar 24, 2021
Implementing changes agreed to resolve issue #266 and in partial resolution of issue #258.
@nathanDtech
Copy link

Will there be any guidance to version support within the LCF compliance?

I recently used an LMS which supports LCF 1.0 and 1.2, but 1.2 only supports fetching and adding entities. An entity can only be modified by requesting a PUT to /lcf/1.0/. PUT requests to /lcf/1.2/ return 405. I feel allowing this may cause confusion to clients regarding the required entity schema, especially client implementations that need to support different LMSs.

@franciscave
Copy link
Collaborator Author

Good question! My recollection is that we deliberately didn't change the endpoints for requests in LCF 1.2, so that they would continue to include /lcf/1.0/. I think that the reason for that was to avoid forcing implementations to change the endpoints every time will issue a minor version update to LCF, since these should be backwards-compatible with earlier versions. If we went to LCF v2.x.x, that would probably require a change in endpoints. @mdovey or @anthonywhitford might wish to comment on this.

The requirement that we are going to make on servers is that, from LCF 1.2.0 onwards, they include a custom header field that specifies the version of LCF supported by the server. My intention is to document this in the profiles specification.

@nathanDtech
Copy link

I suppose the question is whether or not compliance will be tested on a per version basis.

Do you think it might be worthwhile to add a version request? This could indicate which requests/compliance profiles are supported, the version of LCF which is expected for each request (for concurrent LCF versions like above) and possibly whether or not terminal/patron authorisation is required for each request. I think it would be good to have a way of knowing this information ahead of time instead of attempting requests and looking for specific errors or headers.

Interested to hear your thoughts!

@franciscave
Copy link
Collaborator Author

If the server has implemented LCF v1.2.0, any request will elicit a response that contains the custom header lcf-version: 1.2.0.

I like your suggestion that, once we have introduced compliance profiles, there should be a mechanism for reporting which profiles are supported. On the server side that could be done by modification of the custom header, e.g. lcf-version: 1.x.x+P01.1 (I'll have to check syntax constraints in customer header fields).

Should there also be some way for a terminal app to communicate its own compliance? That strikes me as potentially useful if the app is running on a patron's mobile phone (alongside the app authentication that was mentioned in the PlugFest).

@nathanDtech
Copy link

No cases come to mind where transmitting the client's compliance profile would be useful since it's up to the client which requests it makes. There could be scope for sanity checking the requests made by certain profiles and perhaps saving data for unnecessary fields. What do you think the server could do differently if it knew the client profile?

I think there might need to be a bit more clarity on what the LCF version in the URI means. Like in the example implementation above, some calls can be made to /lcf/1.2/ while others have to be made to /lcf/1.0/. Surely this breaks the standardisation because from what you're saying, it sounds like LMSs should use /lcf/1.0/ with the lcf-version 1.2.0 header? I don't think that clients should have to know up front what requests need to be called on what URI for certain LMSs.

I know that JSON isn't officially supported at the moment, but the changes from 1.0 to 1.2 will break JSON serialisation because of the cardinality changes. Will JSON support be retroactive if it's ever supported? It might affect backward-compatibility if it is.

@franciscave
Copy link
Collaborator Author

Sorry for the long silence. I'm not sure that I understand the point about URIs. As far I'm aware, there is no suggestion that the URIs should be modified to contain '/1.2/' instead of '/1.0/'. Using the custom header field to report that the implemented version is '1.2.0' is quite independent of the way that the URIs are constructed. Am I missing something here?

@nathanDtech
Copy link

I feel like we've gone off on a bit of a tangent to my original concern. Sorry if I wasn't clear. I'll try to add more detail...

The LCF server implementation I mentioned claims to support LCF 1.0 and LCF 1.2. Either version can be used by making a request to /lcf/1.0/ or /lcf/1.2/. The problem is that only GET requests can be made to /lcf/1.2/. Any POST or PUT requests to add or modify an entity to /lcf/1.2/ will return a 405. This means that if a client supporting LCF 1.2 wanted to add or modify an entity, it would have to know to go to /lcf/1.0/ instead of /lcf/1.2/ and make the request with LCF 1.0 schema.

I believe this behaviour breaks standardisation. A client shouldn't need to know to make requests to different URLs depending on the method of the request and the implementation it's communicating with. This brings me back to my question of whether or not compliancy should be tested on a per-version basis. If an implementation claims to be 1.0 and 1.2 compliant, should they then support all required requests for both versions, or are you happy for only some requests to be supported on a particular version so long as the others are supported on another version?

If you're proposing to put the LCF version in the header, will that make the version in the URL redundant?

@franciscave
Copy link
Collaborator Author

franciscave commented Jul 13, 2021

Hmm... do we say somewhere that the URL should specify the version number? I thought that '/1.0/' was essentially fixed. I'll need to check this. The version in the URL is in effect redundant, yes... well at least at the moment, but if we were to move to a non-backwards-compatible version 2.0, that would probably make it appropriate to change '/1.0/' to '/2.0/' in URLs.

I've checked the current documentation, including proposed revisions in pull requests, and nowhere do we currently specify '/1.2/' in a URL. Nor do we suggest anywhere that '/1.0/' is not fixed. Perhaps we need to make it clear that it is fixed.

@nathanDtech
Copy link

Thank you for clarifying the versions. Using /2.0/ for breaking changes makes sense to me. I've mostly been using JSON formatting where the updates to cardinality cause breaking changes between 1.0 and 1.2, so I've probably lost some of the context. I'll use XML moving forward to take advantage of the backward-compatibility,

I've just noticed on the 1.2 specification that the version segments of the paths have the comment "LCF version number. All 1.x.x. versions of this specification will use the string “1.0” here." I'm not sure if it was recently added or if I missed it the first time. I recall seeing in the specification somewhere that this segment must be the version of LCF with no further clarification, implying that using /1.2/ for 1.2 implementations was correct. That may have been somewhere in a previous version of the spec though since I can't find it now.

@franciscave
Copy link
Collaborator Author

For changes to implementation profiles see pull request #279.

@mdovey mdovey closed this as completed Oct 11, 2022
@anthonywhitford anthonywhitford linked a pull request Apr 3, 2024 that will close this issue
@anthonywhitford anthonywhitford added this to the 1.3.0 milestone Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants