Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GET request/{request_id} stuck at processing #609

Closed
fliphil opened this issue Apr 4, 2019 · 12 comments
Closed

GET request/{request_id} stuck at processing #609

fliphil opened this issue Apr 4, 2019 · 12 comments

Comments

@fliphil
Copy link
Collaborator

fliphil commented Apr 4, 2019

Hello,

The server appears to be stuck at the "processing" status when my client queries with a GET request.

It moves past the "initial" status to "processing", but after 5 minutes of retries there is no update to the status.

[{"acvVersion":"0.5"},{"url":"/acvp/v1/requests/19","status":"processing"}]

This happens on the first Vendor that try to register.
It fails there because my client attempts to register the Vendors first, but if needed I could shift the order around and try something else such as a Person.

@fliphil fliphil changed the title GET request/{request_id} stuck at processsing GET request/{request_id} stuck at processing Apr 4, 2019
@hbooth
Copy link
Collaborator

hbooth commented Apr 4, 2019

When it reaches the "processing" state the current process has a manual step (read: human) that needs to be performed to validate and accept the data. For demo, I will eventually implement something that will likely auto-approve, or something to approve them in the morning, so folks don't program in something that expects it to happen at line-speed. Once you submit a request, the eventual process will send an email to notify you that your request has been successfully processed. The email notification is likely to be one of the earliest things to get added once we get 1.0 stabilized.

For the moment, send me a note and I can approve any items in the queue if you want to see it advance (I approved the several that I saw just a bit ago).

@hbooth hbooth closed this as completed Apr 4, 2019
@bfussell
Copy link
Collaborator

bfussell commented Apr 4, 2019

Probably missed some discussion on this, but there is a manual(production) or once daily(demo) step required for all new vendors ? How about all other layers of registration such as modules and OEs ?

@bfussell bfussell reopened this Apr 4, 2019
@hbooth
Copy link
Collaborator

hbooth commented Apr 5, 2019

Yes. The data needs to be vetted by something to insure that the input is valid. Currently, that's a human, since that was the fastest way to get this up and running. Validity more or less entails:
Not a duplicate
No harmful input

  • (xss and other forms of input attacks)
  • profanity and other inappropriate uses

We don't expect bad input as anyone who does it will face a loss of their access, but nonetheless we need to have some sort of mitigation.

Auto checking for XSS and other bad input is a tractable problem.
Profanity is not a security issue, but a reputational one, and dictionaries exist (but perhaps not for every language, but hopefully English will be sufficient).

Exact duplicates can be easily managed, but near (or semantic) matches are a tougher problem (I.B.M. vs IBM vs IBM Corp. vs International Business Machines Corporation, etc...).

The expectation is that after the initial flurry of creating the baseline of metadata, that most of the time the existing data will be sufficient with a new product version being added here and there.

It's not yet implemented, but auto-approving a validation will be possible as long as all of the meta-data is already accepted.

@bfussell
Copy link
Collaborator

bfussell commented Apr 5, 2019

Regarding duplicates, here is what I get from GET /vendors. Doesn't seem like that's what is intended.

[ {
"acvVersion" : "0.5"
}, {
"totalCount" : 10652,
"incomplete" : true,
"links" : {
"first" : "/acvp/v1/vendors?offset=0&limit=20",
"next" : "/acvp/v1/vendors?offset=20&limit=20",
"prev" : null,
"last" : "/acvp/v1/vendors?offset=10640&limit=20"
},
"data" : [ {
"url" : "/acvp/v1/vendors/1",
"name" : "STMicroelectronics",
"website" : "www.st.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/1/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/1/addresses/1",
"street1" : "4690 Executive Drive, Suite 200",
"street2" : null,
"street3" : null,
"locality" : "San Diego",
"region" : "San Diego",
"country" : "USA",
"postalCode" : "92130"
} ]
}, {
"url" : "/acvp/v1/vendors/2",
"name" : "Thales e-Security",
"website" : "www.thalesesec.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/2/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/2/addresses/2",
"street1" : "2200 North Commerce Parkway",
"street2" : "Suite 200",
"street3" : null,
"locality" : "Weston",
"region" : "Weston",
"country" : "USA",
"postalCode" : "33326"
} ]
}, {
"url" : "/acvp/v1/vendors/3",
"name" : "Thales e-Security",
"website" : "www.thalesesec.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/3/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/3/addresses/3",
"street1" : "2200 North Commerce Parkway",
"street2" : "Suite 200",
"street3" : null,
"locality" : "Weston",
"region" : "Weston",
"country" : "USA",
"postalCode" : "33326"
} ]
}, {
"url" : "/acvp/v1/vendors/4",
"name" : "Thales e-Security",
"website" : "www.thalesesec.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/4/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/4/addresses/4",
"street1" : "2200 North Commerce Parkway",
"street2" : "Suite 200",
"street3" : null,
"locality" : "Weston",
"region" : "Weston",
"country" : "USA",
"postalCode" : "33326"
} ]
}, {
"url" : "/acvp/v1/vendors/5",
"name" : "Kingston Technology Company, Inc.",
"website" : "www.Kingston.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/5/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/5/addresses/5",
"street1" : "17600 Newhope Street",
"street2" : null,
"street3" : null,
"locality" : "Fountain Valley",
"region" : "Fountain Valley",
"country" : "USA",
"postalCode" : "92708"
} ]
}, {
"url" : "/acvp/v1/vendors/6",
"name" : "AudioCodes",
"website" : "www.audiocodes.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/6/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/6/addresses/6",
"street1" : "1 Hayarden St.",
"street2" : null,
"street3" : null,
"locality" : "Airport St.",
"region" : "Airport St.",
"country" : "Israel",
"postalCode" : "70151"
} ]
}, {
"url" : "/acvp/v1/vendors/7",
"name" : "AudioCodes",
"website" : "www.audiocodes.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/7/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/7/addresses/7",
"street1" : "1 Hayarden St.",
"street2" : null,
"street3" : null,
"locality" : "Airport St.",
"region" : "Airport St.",
"country" : "Israel",
"postalCode" : "70151"
} ]
}, {
"url" : "/acvp/v1/vendors/8",
"name" : "AudioCodes",
"website" : "www.audiocodes.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/8/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/8/addresses/8",
"street1" : "1 Hayarden St.",
"street2" : null,
"street3" : null,
"locality" : "Airport St.",
"region" : "Airport St.",
"country" : "Israel",
"postalCode" : "70151"
} ]
}, {
"url" : "/acvp/v1/vendors/9",
"name" : "AudioCodes",
"website" : "www.audiocodes.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/9/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/9/addresses/9",
"street1" : "1 Hayarden St.",
"street2" : null,
"street3" : null,
"locality" : "Airport St.",
"region" : "Airport St.",
"country" : "Israel",
"postalCode" : "70151"
} ]
}, {
"url" : "/acvp/v1/vendors/10",
"name" : "PGP Corporation",
"website" : "www.pgp.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/10/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/10/addresses/10",
"street1" : "200 Jefferson Dr.",
"street2" : null,
"street3" : null,
"locality" : "Menlo Park",
"region" : "Menlo Park",
"country" : "USA",
"postalCode" : "94025"
} ]
}, {
"url" : "/acvp/v1/vendors/11",
"name" : "Hewlett Packard Enterprise",
"website" : "www.hp.com/go/atalla",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/11/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/11/addresses/11",
"street1" : "19091 Pruneridge Ave.",
"street2" : "MS 4441",
"street3" : null,
"locality" : "Cupertino",
"region" : "Cupertino",
"country" : "USA",
"postalCode" : "95014"
} ]
}, {
"url" : "/acvp/v1/vendors/12",
"name" : "Hewlett Packard Enterprise",
"website" : "www.hp.com/go/atalla",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/12/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/12/addresses/12",
"street1" : "19091 Pruneridge Ave.",
"street2" : "MS 4441",
"street3" : null,
"locality" : "Cupertino",
"region" : "Cupertino",
"country" : "USA",
"postalCode" : "95014"
} ]
}, {
"url" : "/acvp/v1/vendors/13",
"name" : "Hewlett Packard Enterprise",
"website" : "www.hp.com/go/atalla",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/13/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/13/addresses/13",
"street1" : "19091 Pruneridge Ave.",
"street2" : "MS 4441",
"street3" : null,
"locality" : "Cupertino",
"region" : "Cupertino",
"country" : "USA",
"postalCode" : "95014"
} ]
}, {
"url" : "/acvp/v1/vendors/14",
"name" : "McAfee, Inc.",
"website" : "www.mcafee.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/14/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/14/addresses/14",
"street1" : "2821 Mission College Blvd.",
"street2" : null,
"street3" : null,
"locality" : "Santa Clara",
"region" : "Santa Clara",
"country" : "USA",
"postalCode" : "95054"
} ]
}, {
"url" : "/acvp/v1/vendors/15",
"name" : "Motorola Mobility, Inc.",
"website" : "www.motorola.com/Consumers/US-EN/Home",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/15/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/15/addresses/15",
"street1" : "600 North US Highway 45",
"street2" : null,
"street3" : null,
"locality" : "Libertyville",
"region" : "Libertyville",
"country" : "USA",
"postalCode" : "60048-5343"
} ]
}, {
"url" : "/acvp/v1/vendors/16",
"name" : "Motorola Mobility, Inc.",
"website" : "www.motorola.com/Consumers/US-EN/Home",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/16/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/16/addresses/16",
"street1" : "600 North US Highway 45",
"street2" : null,
"street3" : null,
"locality" : "Libertyville",
"region" : "Libertyville",
"country" : "USA",
"postalCode" : "60048-5343"
} ]
}, {
"url" : "/acvp/v1/vendors/17",
"name" : "Motorola Mobility, Inc.",
"website" : "www.motorola.com/Consumers/US-EN/Home",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/17/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/17/addresses/17",
"street1" : "600 North US Highway 45",
"street2" : null,
"street3" : null,
"locality" : "Libertyville",
"region" : "Libertyville",
"country" : "USA",
"postalCode" : "60048-5343"
} ]
}, {
"url" : "/acvp/v1/vendors/18",
"name" : "Fortinet Inc.",
"website" : "www.fortinet.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/18/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/18/addresses/18",
"street1" : "1090 Kifer Road",
"street2" : null,
"street3" : null,
"locality" : "Sunnyvale",
"region" : "Sunnyvale",
"country" : "USA",
"postalCode" : "94086-5301"
} ]
}, {
"url" : "/acvp/v1/vendors/19",
"name" : "Fortinet Inc.",
"website" : "www.fortinet.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/19/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/19/addresses/19",
"street1" : "1090 Kifer Road",
"street2" : null,
"street3" : null,
"locality" : "Sunnyvale",
"region" : "Sunnyvale",
"country" : "USA",
"postalCode" : "94086-5301"
} ]
}, {
"url" : "/acvp/v1/vendors/20",
"name" : "Fortinet Inc.",
"website" : "www.fortinet.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/20/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/20/addresses/20",
"street1" : "1090 Kifer Road",
"street2" : null,
"street3" : null,
"locality" : "Sunnyvale",
"region" : "Sunnyvale",
"country" : "USA",
"postalCode" : "94086-5301"
} ]
} ]
} ]

@hbooth
Copy link
Collaborator

hbooth commented Apr 5, 2019

The data you are receiving is the actual data currently in the database. As I mentioned on the call, cleaning this up is a longer term activity that will still need to be undertaken.

@bfussell
Copy link
Collaborator

bfussell commented Apr 5, 2019

Cleaning up the duplicates is not really our concern :) What I see is a situation where:

  1. Automated has become manual for vendor registration in production and somewhat manual in the demo as well.

  2. vendor/module information must be maintained by the client across sessions if the same vendor/module information is desired otherwise you end up with a new entry. Maybe I'm missing the intent here, are you expecting us to GET the vendor list or module list and locate the one we want used for the validation ? That seems to be a significant overhead for the client.

@smuellerDD
Copy link
Contributor

I guess that is the intent. I am using the following steps with my client:

  1. check in the local vendor definition whether we have an ID on file

  2. if we do have an ID, query that ID and verify the content. If it does not check out, issue a PUT, otherwise go ahead

  3. If we do not have an ID, download chunk-wise the database and compare it with the definition on file. Get the ID for a match.

  4. If we do not have a match, issue a POST

  5. For the PUT/POST commands, we remember the received request ID and query it every time we are invoked to perform a publication operation. Only proceed with the publication if we got an approval.

@bfussell
Copy link
Collaborator

bfussell commented Apr 6, 2019

@hbooth, @fliphil @smuellerDD @atvassilev Exactly my understanding as well. 1-4 is significant overhead for a client. By performing 1-4 operations the client is essentially managing the database for the server. That seems backwards.

For #5, you are saying the client must maintain state, correct ? Even when the PUT is not "realtime" ?

@smuellerDD
Copy link
Contributor

Yes, for all steps we need to retain a state: the ID and the information whether it is a request ID or a real vendor/person/oe/dependencies/module ID.

@celic
Copy link
Collaborator

celic commented Jul 31, 2019

Is this still an issue? As a separate but related fix, we do have the intention of automating the approvals for the demo environment.

@fliphil
Copy link
Collaborator Author

fliphil commented Jul 31, 2019

Will look into this tomorrow to test

@bfussell
Copy link
Collaborator

I've successfully requested a FIPS validation on the demo server and it was later approved so I don't think its an issue any longer.

@fliphil fliphil closed this as completed Aug 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants