Boundlexx API: An API to get all things related to Boundless

Modding
63

Okay, ever one has spoken and the decision is to make a whole new server.

If you actually have interest in helping moderation or anything, please DM me. I certainly do not want sole ownership if a lot of people start using it.

1 Like
64

WARNING:

There will be minor breaking changes in the next update. All of the World Block Color URLs will be changing slightly. The fields that “describe” the colors are completely being changed (currently: is_new_color, exist_on_perm, exist_via_transform, and days_since_last_exo). We are in the process of overhauling how we collect color so help track them better. This is unfortunately completely unavoidable because the way these are calculated are being completely overhauled to make it match how Sovereign world colors work (and to wonderfully simplify the whole process).

The update will hopefully be out tomorrow, but I wanted to post this ASAP in so people were aware before their apps broke.

Below are what the new values will be. Remember, all of these are calculated based on “when the world was created”. So a color that is “new” to a Exo world that appeared last year, may still be on a Sovereign world now.

"is_default",  # is this the default color for the world (always True unless Sovereign)
"is_perm",  # is this color on Perm worlds
"is_sovereign_only",  # is this color on Sovereign worlds (and not Perm worlds)
"is_exo_only",  # is this color _not_ on Perm or Sovereign worlds (Exo only)
"is_new",  # is this a new color to non-Exo worlds
"is_new_exo",  # is a new color for _any_ World
"is_new_transform",  # is a new color to any world (including "by recipe")
"days_since_exo",  # days since last Exo with this color (Exo only colors)
"days_since_transform_exo", # days since last Exo with this color "by recipe" (Exo only colors)
"first_world",  # first non-Exo world that had this color
"last_exo",  # last Exo to have this color
"transform_first_world",  # first non-Exo world to have this color "by recipe"
"transform_last_exo",  # last Exo to have this color "by recipe"
3 Likes
65
  • Color updates mentioned above are live.
  • Added a new endpoint, https://api.boundlexx.app/forum-format/. It can be used to to generate a template for a World to post to the Forums. See Getting a Fancy Forum Template for Your World for more details.
  • Various improvements to Discord Webhook format to match updates for the new Forum template generator
  • Adds average_per_chunk to all Resource Count endpoints to better understand how much of a resource really exists on a world relative to other worlds of different sizes. These will automatically be filled for all new Resource Count/World Polls going forward, but it will take some time to backfill older Resource Counts.
3 Likes
66
  • Adds tracking of colors available for Sovereign worlds. Right now this is exposed via /api/v1/items/{game_id}/sovereign-colors/. Example: https://api.boundlexx.app/api/v1/items/9555/sovereign-colors/. For ease of self usage, you can get a full list Blocks with colors and their IDs at https://api.boundlexx.app/api/v1/items/?has_colors=true&lang=english&format=api
  • Major performance improvements for Polling Worlds. All worlds should be polled again at 10 minute intervals again.
  • Since this has not been posted yet, there are now Discords notifications (via Webhooks) for new Homeworlds, Sovereign worlds and Creative worlds in addition to Exoworlds. These are in use on at least 4 different Discord servers. DM me if you want want to set them up on yours.
  • WIP for uploading World Control data and new Sovereign colors Discord notifications
3 Likes
67

Since it looks like that color data is being collected at irregular intervals, and can change at irregular intervals, is it practical to get a timestamp the data was last verified? It would help apps filter out old data if a world isn’t seen for a while.

68

I already have that data, but I was not planning on exposing it. We already have 4 people running ProxyUI when they play to update colors more reliably. The more people that run it, the better the data will become. My next big task is probably to iterate on ProxyUI to help clean it up and make it more “user friendly” for non-technical folks.

69

Ah, okay. I was just thinking that with dozens to hundreds of these planets being in various states of availability, the limited number of people using the tool are likely to miss some worlds some months, but if you think it will be reliable, no worries.

Edit: In case I was ambiguous, I mean for /worlds/WORLD_ID/block-colors/ more than /items/ITEM_ID/sovereign-colors

1 Like
70
  • Added msgpack serializer. You can now get any of the API responses back as a msgpack response by adding format=msgpack to the URL. While msgpack is not a human readable format (binary), it does make request sizes much smaller! JSON support is not going anywhere, just adding this as well for the new endpoint down below.
  • Adds Blocks API (https://api.boundlexx.app/api/v1/blocks/?format=api) to allow for easy lookup of “block IDs” to items
  • Adds /api/v1/worlds/dump/.
    • This endpoint returns all world data in a single request.
    • As a result, this endpoint is massive.
    • Only supports the msgpack format. Sorry, there is just too much data for this to return it as JSON.
    • Cached for 1 hour.
2 Likes
71
  • Adds html_name and text_name to World API.
    • html_name is a fully resolved HTML name for colors/emojis. Comes with class attributes for easy styling!
    • text_name strips out Emojis and color formatting. Emojis are replaced with the Emoji name (otherwise some worlds would have empty strings for names).
  • Adds sorting by name to World API, add ordering=sort_name or ordering=-sort_name to the URL to sort by the sort_name for a world. sort_name is the same as text_name, but all lowercased and has a database index.
  • Adds is_public_edit and is_public_claim to Worlds API. This will be null (no data) in 99% of cases. You get a world to have this data, it has to be uploaded via ProxyUI.
3 Likes
72
  • Adds 5 requests/s rate limit.
    • Rate limiting is per IP/endpoint combination.
    • Endpoints are defined as they are seen in the API Docs. Different variations of path parameters for a single endpoint (/api/v1/colors/1/ and /api/v1/colors/2/) are still in the same bucket for rate limiting.

I did not want to originally implement rate limiting, but making 800 requests in less then 1 minute is not cool.

2 Likes
73

I hate that! Who was doing that? Request the info, record it to a google sheet or database and only update every so many hours.

800 is excessive!

74

He’s likely talking about me and the fact that http://portal-seekers.com/explorer.html originally made a request for each planet. It would cache these in browser storage for 4 hours but every time someone new would hit the site, it would make 1 request for every planet again. Guilty as charged.

4 Likes
75

Woops. Yea thats why my apps aren’t up yet. Been writing scripts with internal rate limiters on my end :stuck_out_tongue:

76

Is that 5 per minute, per hour or per day?

77

Looks like per second

1 Like
78

I would never query that fast, but if that is the case it is good to know.

79

I had rate limiting on my end as well as browser storage caching of all requests but the rate limit was really low at 40ms between requests. Storage cache of responses was 4h.

1 Like
80

The way the endpoints are laid out, the only way to get the information “what worlds have this color” is to hit all 1000+ world endpoints, which will inevitably cause a flood of requests if you don’t deliberately throttle.

I think that designing an endpoint around the opposite search could be a good way to help mitigate the need for the flood of requests caused by this use case.
If there was a items/item_id/colors/color_id/worlds endpoint, someone could request the /items/item_id/sovereign-colors to get a list of possible colors for an item and then when the color is selected by the user, they could hit items/item_id/colors/color_id/worlds to grab the world list and then grab just the worlds they need to actually display.
I don’t have the mental capacity to actually use the API right now, so this isn’t a request from me, just a suggestion.

An alternative is for the app developer to cache this information and serve it out like that themselves, which isn’t particularly challenging, but does feel redundant.

Note that I wrote this in the very early AM waiting for my son to fall back asleep, so apologies if it’s hard to follow.

81

@Angellus introduced a new dump endpoint that returns all worlds and all blocks colors in one request. Upside? It’s one call and one response so no more 800-1000 hits whenever someone new loads the site. Downside? It’s currently 15MB of data (even with msgpack) that has to be transferred every single time someone hits the site because it’s not cacheable in browser (exceeds browser storage quota).

On modern connections 15MB is not too bad but for mobile users that don’t have unlimited data this could be a problem.

1 Like
82

Yeah, I definitely noticed that change. When the loaded count went away, I pulled up the console to see if the API was broken or something. It feels excessive to have the whole db in my browser, which is why I made that suggestion to limit the data needed at any given time.