Problem parsing JSON #17

Haytam95 · 2018-11-22T12:53:56Z

When parsing a received a JSON from Java (Spring) REST service it doesn't receive nothing. I thought it was because of nulls on Json, so i filled them with empty strings and arrays but still.

Java Entity

@Entity
public class Usuario implements Serializable {

  @Id
  @GeneratedValue(strategy = GenerationType.IDENTITY)
  private Long id;

  @Column
  @NotNull
  private String username;

  @Column
  @NotNull
  private String contrasenia;

  @Column
  private String correo;

  @Column
  private Date fechaAlta;

  @Column
  private Float salud;

  @Column
  private Float blindaje;

  @Column
  private Integer energia;

  @Column
  private Integer saciedadHambre;

  @Column
  private Integer conectado;

  @JoinColumn
  @ManyToOne
  private Crew crew;

  @JoinColumn
  @ManyToOne
  private RangoCrew rangoCrew;


  @JoinColumn
  @OneToMany(fetch = FetchType.LAZY)
  private List<Vehiculo> vehiculoList;

  @JoinColumn
  @OneToMany(fetch = FetchType.LAZY)
  private List<Propiedad> propiedadList;

  @JoinColumn
  @OneToMany(fetch = FetchType.LAZY)
  private List<ItemInventario> itemInventarioList;

  @JoinColumn
  @ManyToMany(fetch = FetchType.LAZY)
  private List<Afeccion> afeccionList;
}

Java REST:

@PostMapping("findUsuarioByUsername")
  public Usuario findUsuarioByUsername(@RequestBody Map<String, String> body) {

    String username = body.get("username");

    return usuarioRepository.findUsuarioByUsername(username);
  }

JSON comming from service (Not null protected):

{
    "username": "Elba_Nanero",
    "contrasenia": "lala",
    "correo": null,
    "fechaAlta": null,
    "salud": null,
    "blindaje": null,
    "energia": null,
    "saciedadHambre": null,
    "conectado": 1,
    "crew": null,
    "rangoCrew": null,
    "vehiculoList": null,
    "propiedadList": null,
    "itemInventarioList": null,
    "afeccionList": null
}

Json null protected

{
    "username": "Elba_Nanero",
    "contrasenia": "lala",
    "correo": "asdasd",
    "fechaAlta": "2018-11-22T13:11:30.996+0000",
    "salud": 0,
    "blindaje": 0,
    "energia": 0,
    "saciedadHambre": 0,
    "conectado": 1,
    "crew": {},
    "rangoCrew": {},
    "vehiculoList": [],
    "propiedadList": [],
    "itemInventarioList": [],
    "afeccionList": []
}

SAMP - Request:

public OnGameModeInit()
{
client = RequestsClient("http://localhost:8080/", RequestHeaders(
        "Content-Type", "application/json",
        "Accept", "application/json"
   	));
    
    new Node:node = JsonObject("username", JsonString("Elba_Nanero"));
    
	RequestJSON(client, "usuario/findUsuarioByUsername", HTTP_METHOD_POST, "OnServerResponse", node, RequestHeaders("Content-Type", "application/json",
        "Accept", "application/json"));
return 1;

}

SAMP - Process Response:

public OnServerResponse(id, status, node) {

	new strResponse[512];
	new strData[128];
	
    JsonGetString(node, "contrasenia", strData);
	format(strResponse, 512, "Recibi: '%s'", strData);
	print(strResponse);
}

SAMP - Error | Empty

A little hand please?

Also, the status code is 2 (WTF)

Update

After some testing, ~~i found out that only strings are the problem~~. Look at this:

~~Float and Integers work fine~~ Neither float and integers are working, they come as "0" always

The text was updated successfully, but these errors were encountered:

Southclaws · 2018-11-22T13:57:00Z

If you have curl available, could you post the output of curling that endpoint with the -v flag? Since nothing seems to be working here, I suspect it's some edge case with the details of the response.

If you don't have curl, instead modify your code to make use of the basic Request function instead of RequestJSON and simply print out the response body and check that it's actually the expected JSON payload.

Also, ensure your headers include a Content-Type: application/json which I would assume they would since Spring should handle that automatically, but it's good to check just in case.

Haytam95 · 2018-11-22T14:18:43Z

I just editted my code, i found something a little strange.

Strings after response data concatenation aren't showing. Maybe encoding problem?

Also, ensure your headers include a Content-Type: application/json which I would assume they would since Spring should handle that automatically, but it's good to check just in case.

Also at Spring, i have:

@RestController
@RequestMapping(value = "/usuario/", produces = "application/json;charset=UTF-8")
public class UsuarioController {

Edit

I just wrote the response in a file...

Southclaws · 2018-11-22T15:16:06Z

That first issue you noticed would be the culprit - the response contains invalid characters. If you open the output file in a hex viewer (Notepad++ can do this I think and Sublime has a plugin for it) you'll see the exact binary values.

But it seems like the problem is with the server, not the client.

Haytam95 · 2018-11-22T17:02:34Z

Just checked characters in response using Postman. It's ok, so i'm out of ideas

Haytam95 · 2018-11-22T17:06:10Z

Also running SAMP default HTTP request, it worked ok:

Southclaws · 2018-11-23T09:40:54Z

Just to make sure: have you seen any error in OnRequestFailure?

Haytam95 · 2018-11-23T11:25:21Z

Noupe, OnRequestFailure is not being called.

And the data gets to Java service, it hit even the breakpoint and all.

Haytam95 · 2018-11-23T11:45:33Z

I was thinking, if there is a way in which we could debug the plugin locally with breakpoints (Using the source code)? Just to see what happens on the response.

I don't know how to program on C++ or develop SAMP Plugins, but if you give me a few hints i'm sure i'll be able to debug it and find out what's going on.

At this point, i'm pretty sure that:

SAMP to Java data is getting fine
Java response is correct and is encoded UTF-8 (tested with Postman)
HTTP native SAMP is getting Java data fine

Do you have something in mind?

Or if you prefer, i can build a .war of my service, give you some endpoints and help you out whatever you need debugging. (It uses Hibernate & JPA, so there is no need to worry about creating databases or populate with data. Just run a Tomcat server and drop it there)

Southclaws · 2018-11-24T19:23:37Z

I noticed you used HTTP_METHOD_POST in the code, but in the Postman and SA:MP standard HTTP call you used a GET method, is this intentional? Ensure your endpoint can deal with receiving data through a POST method if you need to send data to it. If this is the case, there is still something going wrong with the response not raising an error.

Another thing that probably won't help here but is good practice is to check the return value of Request/RequestJSON functions, a non-zero return value indicates an error.

If none of this helps, I'll do some digging tomorrow.

Haytam95 · 2018-11-24T23:18:31Z

I noticed you used HTTP_METHOD_POST in the code, but in the Postman and SA:MP standard HTTP call you used a GET method, is this intentional? Ensure your endpoint can deal with receiving data through a POST method if you need to send data to it. If this is the case, there is still something going wrong with the response not raising an error.

Yeah, it was intentional when i tried default HTTP request SAMP.

Another thing that probably won't help here but is good practice is to check the return value of Request/RequestJSON functions, a non-zero return value indicates an error.

RequestJson is returning 0 and Java service is getting data ok.

If none of this helps, I'll do some digging tomorrow.

As i say, if i can help you with anything just say it!

Southclaws · 2018-11-27T09:22:50Z

Just a quick update to keep you in the loop: I didn't get a chance to look at this at the weekend, took longer than usual to get my Windows laptop set up (not used it in months, had to install Visual Studio and dev tools etc). Will hopefully get time to do this soon.

Haytam95 · 2018-11-27T11:11:05Z

Okay thanks for the update!

Waiting for it!

Haytam95 · 2018-11-29T11:24:36Z

Any news? I know you're probably busy, but i'm depending on this to start a development of a gamemode

As i say, if you are too busy, just give me a hand to set up my development enviroment and i'll try to fix it

Southclaws · 2018-11-29T19:13:22Z

No progress yet - the reason this is taking a while is I build the plugin on a machine I don't own any more, and going through the process of setting up the build was something I neglected to document properly...

As far as I know, it should be a case of installing vcpkg, cpprestsdk, cmake and Visual Studio. I'm currently just awaiting the cpprestsdk installation then I shall try and get the source building.

If you can provide me with a .jar/.war/.exe to run a simple server with some endpoints, it will help with debugging!

Southclaws · 2018-11-29T19:32:04Z

Update: I have the environment set up and I'm ready to debug, hopefully we can get this bug fixed ASAP!

Haytam95 · 2018-11-29T19:33:45Z

As far as I know, it should be a case of installing vcpkg, cpprestsdk, cmake and Visual Studio. I'm currently just awaiting the cpprestsdk installation then I shall try and get the source building.

That's what i tried, but... It didn't go so well, i couldn't make the IDE recognize the libraries and more mayhem.

If you can provide me with a .jar/.war/.exe to run a simple server with some endpoints, it will help with debugging!

Of course! ~~Today i'm full, but i think tomorrow i'll be able to build a war~~

I'll try to make some room in an hour to build a war!

Haytam95 · 2018-11-29T19:35:55Z

Update: I have the environment set up and I'm ready to debug, hopefully we can get this bug fixed ASAP!

Perhaps it's a good time to review the build environment documentation haha

Southclaws · 2018-11-29T19:38:03Z

Yep haha I'm doing that now so I don't forget for next time!

Southclaws · 2018-11-29T20:20:16Z

Documentation updated with development environment setup guide: https://github.com/Southclaws/pawn-requests/blob/master/README.md#development

Southclaws · 2018-11-29T20:31:28Z

So I've realised where the status code 2 is coming from, I encoded error values into the status field when calling OnRequestFailure: https://github.com/Southclaws/pawn-requests/blob/master/src/impl.cpp#L116 but the strange thing is the log should display ERROR: General error: (cause)

Haytam95 · 2018-11-29T23:41:03Z

I'm kind on a rush, so i could make a simple Spring application with some services (Couldn't build the war, tomorrow i have three exams at university and i think they're going to waste me !)

https://github.com/Haytam95/service_test

Just clone & put it to run in Intellij Idea

Haytam95 · 2018-11-29T23:41:39Z

Anyway, tomorrow i'll build the war propertly, sorry!!!

Haytam95 · 2018-12-04T15:32:31Z

Quick update!

I'm having some troubles building the war. For some reason, the service works when running from IDE, but i couldn't deploy the war in a Tomcat.

Did you see the repo (https://github.com/Haytam95/service_test) ? Or are waiting for the war?

I'll try harder to find a solution to the war problem

Southclaws · 2018-12-04T15:38:03Z

Yeah I cloned it and ran it in IntelliJ, I couldn't get it to run as it was throwing build errors (I did resolve them but after that, couldn't get it to run for some other reason. I'm at work currently so I can't give you any more details currently)

Haytam95 · 2018-12-04T16:02:50Z

Don't worry.

It's my mess haha, i'll fix it and improve documentation to make it run.

Once i update everything, i'll leave a comment

Haytam95 · 2018-12-04T16:25:49Z

Ready! So... In a perfect world, if you just pull from the repository, everything should work (At least in intellij. I'm still struggling making a war).

If you run in any trouble, let me know! (I messed up with the POM)

Southclaws · 2018-12-04T17:29:37Z

Thanks, works, I will run some tests on an evening this week (I can't tonight as I'm busy).

Southclaws · 2018-12-05T09:25:01Z

Here's a quick test I ran with curl. First a call to httpbin which is used in the unit tests: https://github.com/Southclaws/pawn-requests/blob/master/test.pwn#L42

➜  ~ curl -v http://httpbin.org/get
*   Trying 34.206.253.53...
* TCP_NODELAY set
* Connected to httpbin.org (34.206.253.53) port 80 (#0)
> GET /get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Server: gunicorn/19.9.0
< Date: Wed, 05 Dec 2018 09:22:48 GMT
< Content-Type: application/json
< Content-Length: 214
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< Via: 1.1 vegur
<
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.54.0"
  },
  "origin": "91.217.237.56",
  "url": "http://httpbin.org/get"
}
* Connection #0 to host httpbin.org left intact

Then a call to the Java app:

➜  ~ curl -v localhost:8080/testGet
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /testGet HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Wed, 05 Dec 2018 09:08:29 GMT
<
* Connection #0 to host localhost left intact
{"id":"asd","name":"asdasdasd","moreData":[]}

Ignoring the request headers and response body, here's the difference between the response headers:

➜  ~ diff -y <(curl -v localhost:8080/testGet 2>&1 | grep '< ') <(curl -v http://httpbin.org/get 2>&1 | grep '< ')
< HTTP/1.1 200  					      |	< HTTP/1.1 200 OK
< Content-Type: application/json;charset=UTF-8  	      |	< Connection: keep-alive
< Transfer-Encoding: chunked    			      |	< Server: gunicorn/19.9.0
< Date: Wed, 05 Dec 2018 09:22:02 GMT   			< Date: Wed, 05 Dec 2018 09:22:02 GMT
							      >	< Content-Type: application/json
							      >	< Content-Length: 214
							      >	< Access-Control-Allow-Origin: *
							      >	< Access-Control-Allow-Credentials: true
							      >	< Via: 1.1 vegur
<       							<

I doubt the missing OK on the status would break it. The keep-alive I'm not sure about. Another thing I'm wondering is whether that Content-Type with the charset directive is something cpprestsdk isn't dealing with properly. There's also the missing Content-Length header which cpprestsdk may require to properly decode the JSON response or something.

However, I still haven't had a chance to properly debug this on the plugin so these are all just assumptions. If there's an issue in cpprestsdk (the library that underpins the plugin) there may not be anything I can do other than wait for a fix.

Haytam95 · 2018-12-05T11:13:54Z

Okay. So this is what i understand: Java is receiving data and cpprestsdk is receiving the response. BUT, it's going trough an exception because "it thinks" something failed, because the headers are different. Am i right?

If they can comunicate, maybe we could try:

Match the headers adding annotations to Java server
Insolate the exception and make it run anyway (if we are certain that the data is not corrupted. I think some information like the charset can be hardcoded, and the length could be scripted)

I'll search about how can i play a little with Spring headers and see how it goes.

I'll wait for more! Good job!

Southclaws · 2018-12-05T11:39:54Z

With Spring and cpprestsdk both being widely used libraries, I would expect both of them to conform to standards and work fine.

Another thing is I think the current binaries are using cpprestsdk v2.10.1 however the Linux builds use the latest pulled from GitHub so Linux vs Windows may behave differently. This may be worth investigating, if the issue doesn't occur on Linux then it's likely fixed in a later version of cpprestsdk.

Haytam95 · 2018-12-05T15:36:28Z

Update:

Just tried including Content-Length to spring headers response. Nothing

New spring service:

Spring new response with headers:

Samp new call to service:

Samp response handling & console:

We can discard that it's because of Content-Length header.

Haytam95 · 2018-12-05T15:40:24Z

Also i just pushed the new service & filter that adds the content-length header.

In case you need to test, it's very easy add new headers.

Haytam95 · 2018-12-10T10:53:53Z

Hi there! How's it going? Any news?

Southclaws · 2018-12-10T19:32:55Z

Unfortunately I can't get the demo server project to run on Windows: java.lang.IllegalStateException: Error processing condition on org.springframework.boot.autoconfigure.context.PropertyPlaceholderAutoConfiguration.propertySourcesPlaceholderConfigurer

Haytam95 · 2018-12-10T19:40:32Z

In the repo, there is the .jar compiled. https://github.com/Haytam95/service_test/tree/master/jar

Just install Java, open a CMD console and

java -jar demo.jar

Southclaws · 2018-12-10T19:49:40Z

Thanks, it's running now.

It seems to be working fine:

*** Test OnGetData

{"id":"asd","name":"asdasdasd","moreData":[]}

PASS!

Using the following code:

#define RUN_TESTS

#include <a_samp>
#include <YSI\y_testing>

#include "requests.inc"

new Request:OnGetData_ID;
main() {
    new RequestsClient:client = RequestsClient("http://localhost:8080", RequestHeaders());
    OnGetData_ID = Request(
        client,
        "testGet",
        HTTP_METHOD_GET,
        "OnGetData",
        .headers = RequestHeaders()
    );
}
forward OnGetData(Request:id, E_HTTP_STATUS:status, data[], dataLen);
public OnGetData(Request:id, E_HTTP_STATUS:status, data[], dataLen) {
    print("*** Test OnGetData\n");

    ASSERT(id == OnGetData_ID);
    ASSERT(status == HTTP_STATUS_OK);
    print(data);

    print("\nPASS!");
}

public OnRequestFailure(Request:id, errorCode, errorMessage[], len) {
    printf("Request %d failed with %d: '%s'", _:id, errorCode, errorMessage);
}

Haytam95 · 2018-12-10T20:11:09Z

Dammit, i literally copied your script and run it in my PC and it still refuses to work.

I tried in three different PC: Two at my work and the notebook on home

Southclaws · 2018-12-10T20:17:48Z

I suppose the latest build has some very minor changes then. I would put this down to a cpprestsdk bug that was fixed in a recent version.

I will provide new binaries for the updated version.

Haytam95 · 2018-12-10T20:20:58Z

Thanks, i'll wait for your update. I hope we could close this issue, it has been a pain in the ass get to the bug hahahaha

Haytam95 · 2018-12-12T18:59:23Z

There was any problems with the new release? Or should just download the lastest? (I don't use SAMPCTL)

Southclaws · 2018-12-12T19:29:35Z

https://github.com/Southclaws/pawn-requests/releases/tag/0.8.5

Southclaws · 2018-12-12T19:29:46Z

Please let me know if this fixes the issue!

Haytam95 · 2018-12-12T20:19:12Z

The plugin doesn't load. There's some kind of trouble with the "entry point of method" bla bla bla

ras0219-msft · 2018-12-13T01:10:59Z

In case it's useful, vcpkg can wrap up a binary zip via the export command:

>vcpkg export --zip cpprestsdk zlib --triplet x64-windows

(other options include 7zip and nuget). The zip contains the cmake toolchain at the same relative path, so if you can build with a vcpkg repo, you can build with a vcpkg export :)

Haytam95 · 2018-12-16T14:51:02Z

@Southclaws in the dependencies folder you left the old cpprest.dll file (cpprest_2_10). I think that's why i'm getting the error.

Could you provide the correct dll please?

Southclaws · 2018-12-17T15:52:33Z

Unfortunately I'm travelling at the moment and don't have my Windows laptop with me. I'm not going to be back until after the new year so maybe someone in the community can do a build for you if it's urgent, sorry!

Southclaws · 2019-01-02T18:45:00Z

I've updated the latest release for Windows - the one that didn't work was built using Boost 66 whereas the bug was fixed in Boost 67. The latest release includes Boost 68.

Haytam95 · 2019-01-02T21:44:57Z

Happy new year! And nice new avatar, by the way

Now it's starting & the request reach fine to the service. BUT

It can't find the public function in the AMX to retrieve the info. It was working previously (When the data was coming corrupted). Something has changed?

I also tried using the native HTTP request (Just to check for misspelling) and it's calling the public.

When a request is created, the address of the
AMX it is called from is stored into the request object which is then
passed on to the response object when the request has finished.

Maybe #20 & #21 solution bring a new problem?

Southclaws · 2019-01-03T08:53:03Z

There are no other changes between versions, all I did was recompile with newer dependencies: 0.8.4...0.8.5

The changes in #21 are on a different branch because it's a pull request, those changes were made later yesterday evening after releasing the binary. I can't reproduce this issue unfortunately but I'll have a look again later.

Southclaws · 2019-01-05T19:50:20Z

0.8.6 has been released which contains the changes for #21 which fixes #20 - it passes all unit tests, let me know if this issue has been resolved.

Haytam95 · 2019-01-06T16:03:40Z

Yeah... But... hahaha just joking, you nailed it!

It was a tough issue, uh? Thank you so much for your time and dedication!

Southclaws · 2019-01-06T16:37:59Z

Great to hear! 🎉

Southclaws closed this as completed Jan 6, 2019

Problem parsing JSON #17

Problem parsing JSON #17

Comments

Haytam95 commented Nov 22, 2018 • edited Loading

Update

Southclaws commented Nov 22, 2018

Haytam95 commented Nov 22, 2018 • edited Loading

Edit

Southclaws commented Nov 22, 2018

Haytam95 commented Nov 22, 2018

Haytam95 commented Nov 22, 2018

Southclaws commented Nov 23, 2018

Haytam95 commented Nov 23, 2018

Haytam95 commented Nov 23, 2018 • edited Loading

Southclaws commented Nov 24, 2018 • edited Loading

Haytam95 commented Nov 24, 2018 • edited Loading

Southclaws commented Nov 27, 2018

Haytam95 commented Nov 27, 2018

Haytam95 commented Nov 29, 2018

Southclaws commented Nov 29, 2018

Southclaws commented Nov 29, 2018

Haytam95 commented Nov 29, 2018 • edited Loading

Haytam95 commented Nov 29, 2018

Southclaws commented Nov 29, 2018

Southclaws commented Nov 29, 2018

Southclaws commented Nov 29, 2018

Haytam95 commented Nov 29, 2018 • edited Loading

Haytam95 commented Nov 29, 2018

Haytam95 commented Dec 4, 2018

Southclaws commented Dec 4, 2018

Haytam95 commented Dec 4, 2018

Haytam95 commented Dec 4, 2018

Southclaws commented Dec 4, 2018

Southclaws commented Dec 5, 2018 • edited Loading

Haytam95 commented Dec 5, 2018 • edited Loading

Southclaws commented Dec 5, 2018

Haytam95 commented Dec 5, 2018 • edited Loading

Haytam95 commented Dec 5, 2018 • edited Loading

Haytam95 commented Dec 10, 2018

Southclaws commented Dec 10, 2018

Haytam95 commented Dec 10, 2018 • edited Loading

Southclaws commented Dec 10, 2018

Haytam95 commented Dec 10, 2018 • edited Loading

Southclaws commented Dec 10, 2018

Haytam95 commented Dec 10, 2018

Haytam95 commented Dec 12, 2018 • edited Loading

Southclaws commented Dec 12, 2018

Southclaws commented Dec 12, 2018

Haytam95 commented Dec 12, 2018

ras0219-msft commented Dec 13, 2018 • edited Loading

Haytam95 commented Dec 16, 2018

Southclaws commented Dec 17, 2018

Southclaws commented Jan 2, 2019

Haytam95 commented Jan 2, 2019 • edited Loading

Southclaws commented Jan 3, 2019

Southclaws commented Jan 5, 2019

Haytam95 commented Jan 6, 2019

Southclaws commented Jan 6, 2019

Haytam95 commented Nov 22, 2018 •

edited

Loading

Haytam95 commented Nov 22, 2018 •

edited

Loading

Haytam95 commented Nov 23, 2018 •

edited

Loading

Southclaws commented Nov 24, 2018 •

edited

Loading

Haytam95 commented Nov 24, 2018 •

edited

Loading

Haytam95 commented Nov 29, 2018 •

edited

Loading

Haytam95 commented Nov 29, 2018 •

edited

Loading

Southclaws commented Dec 5, 2018 •

edited

Loading

Haytam95 commented Dec 5, 2018 •

edited

Loading

Haytam95 commented Dec 5, 2018 •

edited

Loading

Haytam95 commented Dec 5, 2018 •

edited

Loading

Haytam95 commented Dec 10, 2018 •

edited

Loading

Haytam95 commented Dec 10, 2018 •

edited

Loading

Haytam95 commented Dec 12, 2018 •

edited

Loading

ras0219-msft commented Dec 13, 2018 •

edited

Loading

Haytam95 commented Jan 2, 2019 •

edited

Loading