HTTP Shopping API Documentation

Dhusk · March 25, 2020, 9:51am

I started testing this in preparation for the new Portal Seekers website I’m building and am getting a 404 with the following GET:

http://192.168.1.16:8979/api/shopping/S/32805

That IP and port is what’s provided via the apiURL in the list-gameservers response. I have a shop stand with rough oort up. Please halp <3.

Game servers response:

[
   {
      "_lastUpdate":1585129497.82074,
      "addr":"192.168.1.16",
      "apiURL":"http://192.168.1.16:8979/api",
      "atmosphereColor":[
         0.088504999876022339,
         0.94165802001953125,
         2.8768720626831055
      ],
      "chunksURL":"http://192.168.1.16:8979/api/chunk",
      "displayName":"Creative",
      "id":7,
      "info":{
         "afkIdleInterval":450.0,
         "afkIdleThreshold":900.0,
         "maxPlayers":0,
         "players":1,
         "spawnWeight":0.28349599242210388
      },
      "ipAddr":"192.168.1.16",
      "name":"7",
      "numRegions":1,
      "planetsURL":"http://192.168.1.16:8979/api/planet",
      "region":"creative",
      "tier":0,
      "timeOffset":1585128718.560184,
      "waterColor":[
         0.077771000564098358,
         0.31118398904800415,
         0.58382397890090942
      ],
      "websocketURL":"ws://192.168.1.16:8978/websocket/game",
      "worldDescription":"WORLD_TYPE_LUSH",
      "worldSize":192,
      "worldType":"LUSH"
   }
]

Stretchious · March 25, 2020, 11:31am

Are you using the testing branch for your request?

Edit: Although saying that, I’m currently getting a 403 response

Edit 2: just tried using the local loopback address (127.0.0.1) and it’s returning a response.

Mayumichi · March 25, 2020, 11:38am

I missed this before, is there a way to know when the 30min cache expires? I’m planning on letting users re-order my request queue to prioritise stuff they’re actually interested in, but it’d be less useful if the data is just going to be the same with no idea when the next refresh will actually happen.

A http header maybe, max-age or expires?

Also can you elaborate what you mean with “unless other people are doing http requests as well”? Is the rate limit a global limit, not per api key?

Dhusk · March 25, 2020, 11:38am

I’m not. I’m using my local creative server because I thought the direction from @james was to do so until the release was officially released.

EDIT: I’m a super idiot

Stretchious · March 25, 2020, 11:39am

The codebase on testing is different to live - you’ll need to switch to the testing branch before you go into creative.

Dhusk · March 25, 2020, 11:41am

Ah, I see. That makes a lot of sense and now I feel like an idiot. I’ll hop on testing and spin up a creative server. Thanks.

Dhusk · March 25, 2020, 11:47am

Okay I’m also getting a 403. Let me try with an empty API key header just for @#%!& and giggles.

EDIT: Still a 403.

Stretchious · March 25, 2020, 11:51am

Same

Server: Boundless\r\n
Access-Control-Allow-Origin: *\r\n
Content-Type: text/plain\r\n
Content-Length: 0\r\n
Connection: keep-alive\r\n
Cache-Control: no-cache\r\n

Dhusk · March 25, 2020, 12:02pm

Does anyone have a sample response I could use while we wait for this forbidden issue to be resolved or someone to respond with an explanation? Just one response of sample data would unlock me to build out the rest of this part of the site. Thanks!

Mayumichi · March 25, 2020, 12:18pm

I digest it as an integer array so this isn’t really the binary response, but maybe it helps?

Uint8Array(100) [
   18,   0,  77,  97, 108, 108,  34,  41, 44,  40,  34, 73,
  110, 106, 101,  99, 116, 105, 111, 110,  1,   0,   0,  0,
    0,   0,   0,   0, 232,   3,   0,   0,  0,   0,   0,  0,
  250, 252, 251,   0,  16,   6,   0,  77, 97, 108, 108, 32,
   73,  20,   0,   0,   0,   0,   0,   0,  0, 144,   1,  0,
    0,   0,   0,   0,   0,  10, 253,   7,  1,  16,   7,  0,
   77,  97, 108, 108,  32,  73,  73,  10,  0,   0,   0,  0,
    0,   0,   0, 200,   0,   0,   0,   0,  0,   0,   0, 21,
  253,   7,   1,  16
]

I’ve just tested it and I only get 403 when I hit the ratelimit, but that shouldn’t even be on by default.

Dhusk · March 25, 2020, 12:23pm

00010010000000000100110101100001011011000110110000100010001010010010110000101000001000100100100101101110011010100110010101100011011101000110100101101111011011100000000100000000000000000000000000000000000000000000000000000000111010000000001100000000000000000000000000000000000000000000000011111010111111001111101100000000000100000000011000000000010011010110000101101100011011000010000001001001000101000000000000000000000000000000000000000000000000000000000010010000000000010000000000000000000000000000000000000000000000000000101011111101000001110000000100010000000001110000000001001101011000010110110001101100001000000100100101001001000010100000000000000000000000000000000000000000000000000000000011001000000000000000000000000000000000000000000000000000000000000001010111111101000001110000000100010000

That’s your binary string. Thanks for sharing your response!

As for the 403, I’m consistently getting a 403 even when I haven’t requested anything for the last hour.

Stretchious · March 25, 2020, 12:39pm

Odd - I’m only getting the 403 for my first world (ID: 1) … I stopped it and tested on the 2nd world (ID: 2) and I am getting a valid response from this one.

Started the first world again and I still get a 403 response from it (port 8955).

Dhusk · March 25, 2020, 12:39pm

Hmm, nice catch. I’ll give that a shot soon.

Dhusk · March 25, 2020, 1:07pm

@Mayumichi Is the name of your beacon just “Mall” or is it Mall"),("Injection? I’m assuming you’re not in a guild on your creative server as well, correct? Just double checking my conversions here.

Mayumichi · March 25, 2020, 1:09pm

Yes it’s

Mall"),("Injection

I was simply testing how node.js mysql does escaping

The whole thing translates to something like this

[
  {
    beaconName: 'Mall"),("Injection',
    guildName: '',
    qty: 1,
    patrons: 0,
    price: 10,
    coordinates: { x: -774, z: 251, y: 16 }
  },
  {
    beaconName: 'Mall I',
    guildName: '',
    qty: 20,
    patrons: 0,
    price: 4,
    coordinates: { x: -758, z: 263, y: 16 }
  },
  {
    beaconName: 'Mall II',
    guildName: '',
    qty: 10,
    patrons: 0,
    price: 2,
    coordinates: { x: -747, z: 263, y: 16 }
  }
]

Dhusk · March 25, 2020, 1:10pm

Fastest response ever goes to you!

Dhusk · March 25, 2020, 3:16pm

I’m a little confused on something. When I get to item count, meaning I’ve already parsed out and removed from the binary string:

8 bits worth of beacon name length
8 bits worth of guild name length
n * 8 bits worth of beacon name
m * 8 bits worth of guild name

I have this remaining of the binary string:

0000000100000000000000000000000000000000000000000000000000000000111010000000001100000000000000000000000000000000000000000000000011111010111111001111101100000000000100000000011000000000010011010110000101101100011011000010000001001001000101000000000000000000000000000000000000000000000000000000000010010000000000010000000000000000000000000000000000000000000000000000101011111101000001110000000100010000000001110000000001001101011000010110110001101100001000000100100101001001000010100000000000000000000000000000000000000000000000000000000011001000000000000000000000000000000000000000000000000000000000000001010111111101000001110000000100010000

While the first 8 of that give me 1, which is the item counter we’re after, item count is specified as a u32 implying 32 bits.

It’s been a long time since I did comp sci in college and I haven’t had to use any of this in 15 years. I’m trying to do this in Javascript so I’m curious if anyone knows what’s up here. Perhaps it only sends 8 bits for the item count if it fits?

There’s no chance of a more standard JSON response from the apis you guys put out is there @james? lol

@Mayumichi I think I’m overcomplicating this. You said you’re using node which means you’re using js. Do you have a snippet showing how you ingest the API response into an integer array? This is my first time working with an api that returns actual binary.

Mayumichi · March 25, 2020, 3:50pm

The key to my method is DataView so the array type doesn’t really matter, you just need an ArrayBuffer with the data in it to use DataView.

gist.github.com

https://gist.github.com/mayumi7/08f9f30f1fb9dae4055baad488dc6ff3

parser.js

parseShoppingApiResponse(byteArray) {
	let orders = [];
	let byteOffset = 0;
	let dataView = new DataView(byteArray.buffer);
	while(byteOffset < byteArray.byteLength) {
		let beaconNameLength = dataView.getUint8(byteOffset + 0);
		let guildNameLength = dataView.getUint8(byteOffset + 1);
		let order = {};
		order.beaconName = '';
		for(let i = 0; i < beaconNameLength; i++) {

This file has been truncated. show original

edit: Actually getting the response to the ArrayBuffer depends a lot on your js engine, I’m using node.js which has a http library and uses node.js specific things.

gist.github.com

https://gist.github.com/mayumi7/679cea22095e10bab20c13128b149723

httpRequest.js

function httpRequest(url, headers = {}) {
	return new Promise((resolve, reject) => {
		let htObj = (url.startsWith('https') ? https : http);
		let req = htObj.get(url, {headers : headers}, (resp) => {

			// A chunk of data has been recieved
			resp.on('data', (chunk) => {
				data.push(chunk);
			});

This file has been truncated. show original

Dhusk · March 25, 2020, 3:52pm

Oh wow. That’s extremely generous of you. My next area to explore was using ArrayBuffer and parsing out each individual part into the appropriate data type but you just saved me what would likely be several hours of work so thank you!

Happy to repay you in-game if you’d like!

Mayumichi · March 25, 2020, 3:56pm

No problem, don’t worry about it If it helps you I’m glad, it’s most likely not the best solution to the problem but it works.