HTTP Shopping API Documentation

Modding
84

I wasn’t really planning to code to this as all since I’ve never moved forward on any boundless-related web property, and it all sounds rather cumbersome. I don’t have a VPS to dedicate or any desire to set up a 24/7 scanner to scrape this API and store to a web server over my home connection, and it doesn’t sound like any sort of on the fly client (or server) side scripting is going to be very useful with this.

Still, I was thinking about some hypotheticals.

Perhaps there’s some thought that the CDN layer would make this more feasible, say a AJAX-style javascript on a page that attempts to get a list of items on demand from a user without building a local dataset to feed your own front end with.

  • Does the CDN recognize or respect API keys at all? Can an application make requests and retrieve cached data without the world server validating the request?

I don’t see any information here about over limit, invalid key, or other error codes/responses. Not knowing if the CDN is doing any request or response validation:

  • Can a user flood the CDN cache with error responses?

  • Do invalid/over-limit/unauthorized requests trigger the rate limit?

I guess I’m curious whether a malicious, or just poorly coded, application can either flood the cache with error responses, lock up the server with timeouts, or both.

Also whether or not an AJAX type web scanner can become fully or partly functional by relying on parties running scanning daemons to keep the cache populated most of the time.

A central server that authorized (wonderstruck hosted) world servers pushed to on update would open up some interesting possibilities, not limited to shopping data.

Making this configurable would allow private worlds and creative worlds to integrate with privately hosted web interfaces and applications as well.

Population data, and other items could lead to interesting possibilities for social integration apps/plugins possibly even going as far as weather reporting or… IDK what sort of other info people would be interested in. Or what the world servers could easily provide, with this sort of architecture in place.

Certainly most of what people are currently pulling from the discovery servers, or other interfaces.

Just fantasizing :slight_smile: Feel free to ignore the CDN/API questions if I’m not making sense, as noted they’re not related to any active development from my end.

85

So just to be clear every API user will be able to do 1 request per second, the first request after the cache has elapsed will refresh the data?

86

I suspect the current implementation is how it is simply because it was the path of least resistance, nothing major had to be built. Maybe some day we’ll get an upgrade with a more centralized approach, but that’s not a simple thing to do. Now they just exposed the same data the game client is already querying to populate the price list and made it queryable without an active game connection but with the API key instead.

87

The other advantage of the API working directly against the worlds is that it then also works against the Creative and offline worlds. So it’s much more portable.

3 Likes
88

Reading between the lines: Boundless’s Gold Days are just ahead!!!

1 Like
89

I might have missed, apologize if i did.

Rate limits?
I not want to over tax the api. Would be awesome if i could just rate limit cycle all the items on all the planets 1 per hour. Aka spread out requests so before cycle gets back to item 1 its been 1 hour. Should be atleast once or twice a 24 period i would hope.

90

For some, might ask what todo with this?

If ur one that likes analytics and playing with stock market. This is all the fun for u.

Idea is pull data from planets then have a computer program run a buy and sell comparison. Then find your portal routes. Make money by buying and selling! Cut out the middle shop stand man with hand trades. Get with suppliers to buy out and then then move to the buys. Make money with the logistics.

91

Still some questions about the CDN and throttling.

Do I understand correctly that the game server itself has a global 1-sec “cooldown”? It’s not per API key, but global?

What happens if two users request a different non-cached resource (so the CDN doesn’t help) from the same server at the same time? The lucky one gets through and the other one gets a 403 response?

On the other hand the CDN apparently doesn’t throttle, so if I’m hitting cached responses I could theoretically skip the 1-sec delay and only wait if I hit a fresh response?

92

From my understanding…

  • Global
  • The lucky one gets through and the other one gets a 403 response.
  • Yes

I guess we need to use it in earnest to see where any issues may arise.

93

@Mayumichi i do think for a website like yours you only need a refresh every 15 or 30 min. Every second is a bit overrated anyways.

Will there be any limited access per api key or something that will block it if someome decide to troll it a bit and ask every seconds @lucadeltodecso

94

Yeah I’d be really happy with a 15 minute refresh rate, atm it looks like it’ll be maybe once an hour even at 1 request per second so hammering it as fast as possible.

@Stretchious Yeah but that’d mean that every user will essentially double the required delay between requests (IF they want to determine the order in which the items are updated). And if even one user doesn’t have some kind of configurable “user count” that has to be kept updated with known API users, they could hog the whole system to themselves. I’d have no idea in which order they’re going through the item list, or in the worst case scenario they’re only scanning a specific subset of items so some might not be updated at all.

1 Like
95

If z is being used for a 2D coordinate, what is being used for what cartographers usually call z (the 3rd dimension, like elevation or height (can be other things for various spatial/network analysis etc).

96

Y is altitude in Boundless.

97

Oh I’m planning a distributed Maoist performance art collective that’ll be using it to pass on forbidden knowledge, and setting up a timed task list that’ll use the quantitative data to communicate very nefarious things on the sly. Of course we’ll maintain a traditional cash shop as a cover.

1 Like
98

Every single item is a separate request. Once for the shop stands and once for the request baskets. So there’s what, 1000+ items?

It’s not like you can just pull ‘the list’ it will take something like 40 minutes, best case, to get the info for one planet.

1 Like
99

Oh i see.

Maybe something on their side could be done by making a list of every shopstand and request basket per planet, then an api to retrieve this data would be “better”.

1 Like
100

So one thing that I’m either misunderstanding or is being overlooked is that these APIs only request a single item, of a single type (stand/basket) for a single planet at a time.

Is the request limit restriced to each planet API or the overall API?

If it’s the overall, it could take nearly 31 hours to request every item from every planet.
It it’s just per planet’s API, down to ~40 minutes…

101

This is my impression.

There is no “universe” in this system. Each planet is basically a mirror of the same code with some swapped up config files.

To gather all pricing data you will require

(# of items in game) x (2 or more seconds) x (# of planets in game) = # of seconds for a single scan

You may be able to accelerate this a little if you sort your searches by item rather than planet, but it’s still going to be thousands of seconds (100 queries x # of items), best case.

102

@james @lucadeltodecso
Does the in-game use of the shop scanner add that item to the cache for the API users?
Like, if I go to Alder and seach for Bitter Beans in-game and then use the API to make the same request, will I get the cached data or will it be a fresh request (assuming no previously cached API data)?

103

the cache isnt in the server, so no, no interaction between in-game queries and external queries.