ApiOpenStudio Production Docker video

We have released a new video on YouTube, demonstrating the speed and ease of deployment: spinning-up full running API and Admin installations on separate servers in under 30 minutes!

The ApiOpenStudio Production Docker video can be found at https://www.youtube.com/watch?v=iZ_Q81MhXUw.

This is a response to the discovery of an issue with the admin Docker image at the end of last week. The issue was resolved over the weekend and the docker images have been re-published.

ApiOpenStudio Admin production docker fixed

While preparing for yesterday’s seminar, it was found that the naala89/apiopenstudio_admin docker images were broken (the issue can be viewed here).

The good old fashioned “it works on my machine” came back to bite us, but the code has been updated and it is now loomed correctly into the GitLab pipelines. Many thanks to laughing_man77 for jumping on this so quickly!

Problem solved!

We’re pleased to announce that the issue has been resolved, and all tags have been re-uploaded to docker hub. So that was a really quick turn-around of 1 day (and we even had a chance to sleep on it before pushing it live).

Note: the only code that was affected was the https://gitlab.com/apiopenstudio/docker_images/apiopenstudio_admin_docker_prod repository. So if you have an existing checkout of the admin MVP code, this will not affect you.

Apart from the good news that full production images for ApiOpenStudio core and admin are fully working and tested, I also tested the released image on an existing server – and proved that the install time of admin is under 5 minutes!

Emerging new job titles in LCNC an indicator of profound change

What do we call the users of Low-Code or No-Code platforms that work for the IT dept?

Across the board, business and IT is seeing a transformation that is being driven by Low-Code and No-Code technologies. There is clear change in the job titles around development although the new terms have not yet stabilised.

Historically problems like this, are the indicator of genuine profound change!

Low-Code and No-Code is definitely not a replacement for developers or an IT department, and we will never be out of jobs (who will develop the Low-Code tools, who will design and build the surrounding complex architecture?). However, there are some great side-effects:

  • Business employees are learning the basics of
    • Development processes – the how-to and why they are beneficial, as Low/No-Code processes need to be constrained to stop ballooning and undocumented code.
    • Code and processes are more easily aligning with business processes.
    • Programmatic thought.

This is an exciting time, not only because we are seeing those at the forefront of the new technology explosion see their ideas come to fruition, but also because IT departments will see more understanding from business (as they learn more about IT as they onboard with the LCNC applications and processes), and this is driven by business and economics.

The language problem

Across many eras in IT there seems to be one thing that stands out about true – long lasting and permanent changes to the IT landscape break the language used in the current IT ecosystem. So any “language break-down” emerging is evidence of a strong of a probable trend being confirmed as profound change!

Some changes are fast and have “epoch moments”. Some are slow and creep up on us, leading us to say “this has always been known, but we never said it before in that way”.

Economists note that the one thing that stands out about genuine large scale change, is that the new activity will change what people do, how they are specialised and the job titles that describe it.

A good example of this is web-design. Initially we just had “web-designer”, before shattering that role into multiple disciplines, as we came to understand that improving the interfaces on a website was inherently complex, and architecture was only half the problem. We now have roles like (Sven Jenzer, 2023, Typical UX disciplines, [ONLINE] Available at: https://en.wikipedia.org/wiki/Parenthetical_referencing, Accessed 27 January 2023):

  • User Research
  • Information Architecture
  • Interaction Design
  • Usability Evaluation
  • Accessibility Evaluation
  • Visual Design

We need new job titles in development, so that Low-Code enabled developers can be classified in a way that reflects any extra value that they bring. While distinguishing them from citizen developers who, whilst they are not IT people, are doing ad hoc IT tasks to improve their role and also need a way of classifying their extra skills.

What is the solution?

Let’s look at the typical range of developer roles that we currently have:

  • IT Manager
  • Technical Architect
  • Technical Lead
  • Senior Developer
  • Developer
  • Junior Developer
  • Trainee Developer

We currently have no way of expressing the extra values that trained Low-Code developers and No-Code citizen developers bring. So, let’s break this down, essentially we are trying to express:

  • Experience level
  • Value to the business
  • Potential leadership role
  • Area of expertise

Using these taxonomies, we could potentially create new titles to express these, such as ApiOpenStudio Developer, Senior RapidAPI Developer, Junior Zapier Architect. However, this does beg the question, are users of Zapier or RapidAPI developers or architects? To be honest, not really, and they are definitely not architects. So this will not suffice.

I’m not posing a solution to this yet, and just want to open the floor to a problem that is likely to surface very soon, when those that have spent the time learning these skills want to express their new-found knowledge recognised by the business, or businesses want to find workers with the assets and skills that they want.

API First Concepts and Problems

Introduction

APIs are displaying continued astronomical growth (see https://www.postman.com/state-of-api/api-global-growth/#api-global-growth) and are now a critical part of nearly all modern architecture. However, we have seen a lot of coverage recently of the different terms encompassed by the API First methodology, especially as people try to use these buzz-words to generate excitement over their products without fully understanding what they mean.

In IT, terms tend to evolve, due to the pace of change. But this is not the ideal way to create terms as it can lead to similar terms, causing sub-optimal communication, even mistakes. But unlike industries like aviation and maritime, which are the gold standard for communication, w do not set a standard and declaration for what precise terms to use and when, to prevent confusion, misunderstanding and planes and boats crashing into each other.

For instance, we now have the following:

(Yes, I know that some of these are unrelated to APIs, however there is a real problem with language here – everything sounds the same and confuses people)

  • “API First”
  • “Code First”
  • “Data First”
  • “Design First”
  • “API First Design”
  • “Business First API Design”
  • “Api First Culture”
  • “Api First Company”
  • “Api Strategy”
  • “API as a Product”
  • … and the list goes on…

I don’t know about you, but my head is rattling from all of these variants. This is getting ridiculous and incredibly confusing for everyone, requiring a lot of reading of articles to understand the nuances of the new terms that are being introducing and how these may subtly differ from the others.

As technologists, we need the business community to fully grasp the ideas and benefits behind the API first methodology so that we and they can make sound judgements on directions and strategies. This means that the ideas need to be immediately comprehensible and their ramifications obvious.

Rubbish pile
Image: https://www.onmanorama.com/lifestyle/news/2022/08/10/eliminate-impact-single-use-plastic-products.html

I am going to suggest that we throw out a lot of these phrases and keep things simple, but I won’t try to reinvent the wheel and create completely new phrases. For the rest of this article, I will only refer to 3 terms.

API First concepts

Each term has an impact on the technical and business strategies/successes in different measures. But each concept requires a full “buy-in” and understanding from the business.

API First

  • Design the and build the API first!
  • The product is built around the API.
  • This is reflected in the business strategies and revenue models (see API as a product and API design 1st).

API-first is obvious from the business point of view. The API is the primary income generator or essential to other income product/s. Without a revenue model, you do not have a business. Therefore if APIs are the main product or critical to a product that is being created/overhauled, then the API becomes a crucial first-class citizen, not just for the technical dept, but also from the business.

Information and data are key to the modern business, right? APIs are the backbone for nearly all data communication.

Building a new product (not just API) is an expensive task, both in time and money. By going through a full design phase for the API, the correct emphasis and attention is given to the API that it needs, a lot of thought will be put into:

  • What data is being shared across the API.
  • Why this data sharing is required or wanted.
  • Who/what will be consuming the data (internal and/or external).
    • This will reveal any potential flaws or opportunities in assumptions that the business has made.
  • Ensuring that all business processes are made more efficient by the new agile data flow that APIs will provide.
  • Performance.
  • Reliability.
  • Scalability
  • Usability.
  • pricing.
  • Future changes/versioning.

n return, API First will have a causal effect on the business by ensuring that due diligence and research is followed before embarking on a costly venture of system build. A lot of thought will be put into what the business needs and why. Future projections and plans become much clearer.

This does not mean that all other resources sit in idle-mode until the API is built, in fact it can happen simultaneously. Once you have the API model built in documentation like OpenAPI or Redoc (quite early in the process), you have a model that can be used by consumers before you start the full development. This saves time and money on fixing broken assumptions, missing or inadequate requirements.

API as a Product

  1. The API may be the core product, but not necessarily.
  2. API as a product has 2 definitions:
    1. External-facing API that is monetised
    2. Indirect monetisation by a product that requires an API to function or public exposure and usage through a free public API.

API as a product, is tightly coupled to the API design-first concept. In order to implement business strategies for revenue driving APIs, good design and documentation is a “must-have” to enable third parties to consume the APIs, providing an easy-to-use, comprehensive and unified experience.

If the API product is well designed and documented, this will increase the consumption and efficiency of the API (and therefore income). Postman alone has seen very significant increases in traffic and API usage, despite the global downturn and Covid:

Since the 2021 State of the API report was released, the Postman API Platform saw significant surges in use:
* Users: 20 million
* Collections created: 38 million
* Requests created: 1.13 billion

https://www.postman.com/state-of-api/api-global-growth/#api-global-growth

Api First Design

In general, this is a development model for APIs that ensures the following criteria are met:

  • Reach agreement from all stakeholders.
  • Design the APIs models first on OpenAPI or similar product.
  • Build the API.
  • Separate products can be built at the same time as the API.

API design first may not be quite so obvious from the business POV, because at first view, it is just a development process. However, because it requires approval from all stakeholders, it is critical to have business “buy-in” to the process. This enables all of the business to see what/why the API is being built, and ensures that everyone is agreed on the most critical features they need.

In addition, this results in a rare instance of a complete understanding of what is being built and why, by the technical department. Instead of just being told to “build this” by the business without any consultation between them on what the rationale is, what is technically feasible, or if there is a better solution, the developers and department see the vision and are able to work towards that vision in a much more efficient manner and the end result will be a much better product.

Conclusion

This all ties in with discoveries and revelations being made about “Developer Experience”.

Up until recently, a lot of research and time has been invested un user experience (UX) and there is no denying the value that this can contribute to the business by increasing traffic and revenue. However a new key realisation is being made about developer experience (DevX). Who will be directly interacting with and consuming the APIs (and therefore making critical decisions that will impact their business about what APIs to consume and the frequency)? That right, developers!

They will correctly discard badly documented or difficult to use APIs in favour of better APIs. This means your product will be directly impacted by the DevX experience!

If you are supplying an external facing API for revenue (direct with monetisation or indirectly by offering APIs free with the associated traffic, publicity and exposure), then DevX is crucial to maximise income.

If you are supplying an internal facing API, then good DevX will have an impact on associated systems design, development and maintenance. Business processes that interact with the API will be either made more efficient by a well designed API or made cumbersome slow and inefficient by bad APIs.

Low code trends and the new workflow management

The recent flood of articles discussing the way that Low-Code is changing the landscape of development have 2 key elements:

  • Low-Code is now clearly emerging as one of those enormous waves in tech, that will change the way that things are done across business and IT development moving forward. For both tech companies and non-tech companies (because everybody is technology driven).
  • In the future, it is really how these tools are used in unison with the company’s “Pro-Code” developers. Which is the real business problem to be riffed on and solved! 

What does the new development environment that includes “Citizen Developers” look like? How will they interact with IT Departments and the Pro-Code developers?

Will Low-code replace Pro-code?

Up front we’d like to point out the answer here is something we are sure of. This will in no way reduce the demand or work for Pro-Code developers.

There is compelling economic reasoning for this. As while it is true that all companies want to save costs, we are in an increasing world of digital transformation, where every business is in reality, a tech business. 

So the one area where you cannot start with cost saving, is technology. Companies will always be in a technology race to the leading/bleeding edge with their competition. That edge will always require having the best Pro-Code people you can get, as they provide the competitive advantage. 

Paradoxically this means if you have that, you also need the best Low-Code resources for two reasons:

  1. Low-Code enhances the output of your Pro-Code work force, doing vital but repetitive, non-challenging tasks. Speed means less hours, more output, and we all get that.
  2. These are considerable the other benefits, beyond cost savings. One of which is the morale & job satisfaction of the Dev team. Developers universally report that in their ideal work environments, they want to be working on important and challenging tasks. Where they have the best chance of learning things along the way.

So Low-Code can reduce the time they spend on these less interesting tasks. Because you can’t make the tasks go away. The answer is to use a tool that lets you get more done in less time.

More accuracy and happier teams

The hidden part of all this is the organisation’s abilities to utilise No-Code/Low-Code and adapt to it.

In terms of moral & satisfaction, there is clear evidence that the most onerous of tasks are the organisational ones. Like storing names & levels of permission on an API, in the case of ApiOpenStudio. These often have a high cognitive load because they require high concentration but they are in no way interesting. Low-Code solutions come with User Interfaces and Dashboards and these reduce cognitive load and risk in these tasks by making the tasks simpler and providing ways of validating your entries are correct and making the editing/deleting of changes easy. So there are huge benefits in day-to-day operations.

Learning a new technology and skill can also be fun. As we know, the primary driver for nearly all employees and their job satisfaction are:

  • Confidence in the leadership team.
    • Implementing Low-Code in a balanced way will demonstrate that the leadership have a forward-thinking mentality.
  • The employee is valued.
    • Providing training in the new technology and processes will show that the business wants to keep them and invest them.
  • Satisfaction with daily tasks.
    • The employee will feel more satisfaction through more responsibility and seeing their work being visibly used.
  • Learning and growth.
    • We all want to grow and extend out knowledge and usefulness, with the additional benefits of making us more desirable to employees.

(InfoSurv Research 2012, What Are the 4 Top Drivers of Employee Satisfaction in 2012?, viewed 17 October 2021, https://www.infosurv.com/what-are-the-4-top-drivers-of-employee-satisfaction-in-2012/):

Governance

With the decision to introduce Low-Code platforms, your first issue should be governance and compliance.

Your “Citizen Developers” and Project Managers should start to deepen their communication and feedback loops. Most importantly, non-technical people have to really feel they can come to the experts with the most basic questions, even if it is just to check they are on track. This is all new to them and they are not professionals in this sphere.

Because deep, first-hand knowledge of requirements is another advantage that citizen developers brings to the table. It’s worth noting that their approach will usually be decided on the functionality that they want. The effects on governance or the rest of the architecture are normally the furthest from their minds.

So from a Project Management perspective, clear leadership is needed right at the start in setting out the processes needed on the project. So that features and functionality are plumbed in properly.

Communication

There needs to be clear communication between the citizen and Pro-Code developers (usually via the Project Manager), but this can also be direct. This will ensure that any requirements between the two teams are met and no-one is adding useless things to the system, or features that will slow down the system performance.

Once you have initiated a communications format between citizen developers and the development team, then testing and documentation can be mapped out.

Documentation

A clearly written and easy to access set of documents will be required by both your citizen developers and all levels of the development team.

This will ensure that:

  • Onboarding is quick and easy for citizen developers and they can quickly find answers to any questions that they have, without feeling overloaded by technical answers.
  • Managers and project management have access to sources of truth that enable them to answer any questions.
  • The Pro-Code developers are aware of the Low-Code systems and how they interact with their work.
  • Leaders of the development team can easily find a broad overview of the different systems, the interaction between them and the processes that have been implemented to govern them.

Quality assurance

Depending on the complexity of the No-Code solutions and their impact on the rest of the technology, it is very likely that extra guard-rails will have to be introduced. This will probably involve QA (quality assurance), the ability to import and export configuration, and the release cycles.

Even if the output of the citizen developers is content (for example: pages in a blog or CMS), there should be systems in place to accomodate proof reading and validating the look and feel on different devices. There are additional complexities here, if the content that the citizen developers create is all in a database and cannot be imported/exported – where will this be tested, how will it be released? Although this issue is usually solved by that specific technology’s community, time should be spent looking at the different solutions proposed to find the best solution for your organisation.

In the case of complex No-Code solutions that interact with the output from the development team, then these need to go through the full QA feedback loop. QA can then validate that the existing functionality is not broken and that the new functionality works as expected and different systems interact as expected.

What is important, is that that the development team should reach out to the management of the Citizen Developers, so that they can have the discussion with them about compliance and governance & how they must comply with the IT departments direction (and vice versa). This is often an issue in companies, where people directly report in their department, and are in effect silo’d from the other departments.

Moving into the era of Low-Code/No-Code enhanced Companies. 

The organisations that are going to maximise the benefits of this new wave, should already be initiating broad company-wide conversations. These will range deeply across all levels of the organisation. From the way that Pro-Coder work days and productivity will be enhanced to evaluating the advantages and disadvantages of implementing Low-Code solutions for specific tasks.

This will even permeate as far as hiring & the HR Dept. As you look to harvest the advantages of onboarding people in traditional roles that have these additional skills, or incentives for current employees who wish to acquire No-Code or even Low-Code capability. 

This is the conversation of the future for all companies that will succeed in this new era!

A Deep Dive into JWT Tokens in ApiOpenStudio Auth

Introduction

After a ton of work, over the next week or two we will be introducing JWT tokens for authentication in ApiOpenStudio.

This will:

  • Significantly increase the speed of resource requests.
  • Make individual transactions stateless.
  • Maintain the granular access rights to resources, based on the user’s access rights, the resource’s holding account & application, and of course the resource itself.
  • Ensure viability and ease for enterprise clients to use 3rd party authorisation services.

This article will take a look into the JWT specification, current practices and how it is used in ApiOpenStudio.

Rationale

While trying to optimise the authentication DB queries that are performed before the resource was processed, we came to the realisation that the queries were quite long and involved the joining of multiple tables and had several sub-queries so that the query could take into account our extensive range of access rights, like Administrators, Account managers, Applications Managers, Developers and Consumers. Although this would obviously be faster in a production environment, this was causing 1-2 seconds additional processing time to calls in development environments…

Because this had to be calculated every time a resource was called!

The solution was to introduce stateless authorisation, in the form of JWT tokens.

JWT is rapidly becoming the industry standard for authentication and JWT tokens have what are called Claims, which are individual name/value pairs within the body of the token and the token itself is encrypted and secure, which means that sensitive data can be securely included in the token and thus means that the user’s roles and permissions can be included as a claim and only need to be fetched once (during the JWT token GET call).

In addition, JWT tokens have a TTL, which means that we do not need to store a bearer token in the DB against each user (use that to fetch the user that each time a request is received) – because if the token is valid, then there is no need to fetch the user and check the bearer token TTL.

Here are some scenarios where JSON Web Tokens are useful:

Authorisation: This is the most common scenario for using JWT. Once the user is logged in, each subsequent request will include the JWT, allowing the user to access routes, services, and resources that are permitted with that token. Single Sign On is a feature that widely uses JWT nowadays, because of its small overhead and its ability to be easily used across different domains.

Information Exchange: JSON Web Tokens are a good way of securely transmitting information between parties. Because JWTs can be signed—for example, using public/private key pairs—you can be sure the senders are who they say they are. Additionally, as the signature is calculated using the header and the payload, you can also verify that the content hasn’t been tampered with.

auth0.com, JSON Web Token Introduction – jwt.io, viewed 1 September 2021, https://jwt.io/introduction.

How does JWT work?

Pronunciation

Before we start, for a bit of fun I’d like to set the record straight. I’m not one of those boring “absolutists” who insist that GIF should be pronounced “JIF”, even though the original creator of the tech obviously could not spell when he declared that was the pronunciation. But many people are driving me bananas, by mispronouncing JWT: just pronounce it as “JOT”, it’s also easier to say than the most common variant “Jay-Dubbyah-Tee” (this is not a relative to a former US president).

The suggested pronunciation of JWT is the same as the English word “jot”.

Jones M, Microsoft, Bradley J, Ping Identity, Sakimura N, NRI 2020, JSON Web Token (JWT), Internet Engineering Task Force (IETF),viewed 1 September 2021, https://datatracker.ietf.org/doc/html/rfc7519#section-1.

JWT overview

Now for the technical stuff…

JWTs represent a set of claims as a JSON object that is encoded in a
JWS and/or JWE structure. This JSON object is the JWT Claims Set.
As per Section 4 of RFC 7159 [RFC7159], the JSON object consists of
zero or more name/value pairs (or members), where the names are
strings and the values are arbitrary JSON values. These members are
the claims represented by the JWT. This JSON object MAY contain
whitespace and/or line breaks before or after any JSON values or
structural characters, in accordance with Section 2 of RFC 7159
[RFC7159].


The member names within the JWT Claims Set are referred to as Claim
Names. The corresponding values are referred to as Claim Values.
The contents of the JOSE Header describe the cryptographic operations
applied to the JWT Claims Set. If the JOSE Header is for a JWS, the
JWT is represented as a JWS and the claims are digitally signed or
MACed, with the JWT Claims Set being the JWS Payload. If the JOSE
Header is for a JWE, the JWT is represented as a JWE and the claims
are encrypted, with the JWT Claims Set being the plaintext encrypted
by the JWE. A JWT may be enclosed in another JWE or JWS structure to
create a Nested JWT, enabling nested signing and encryption to be
performed.


A JWT is represented as a sequence of URL-safe parts separated by
period (‘.’) characters. Each part contains a base64url-encoded
value. The number of parts in the JWT is dependent upon the
representation of the resulting JWS using the JWS Compact
Serialization or JWE using the JWE Compact Serialization.

Jones M, Microsoft, Bradley J, Ping Identity, Sakimura N, NRI 2020, JSON Web Token (JWT), Internet Engineering Task Force (IETF),viewed 1 September 2021, https://datatracker.ietf.org/doc/html/rfc7519#section-3

What does this mean?

A token is comprised of 3 parts, the header, payload and signature.

The header of the token defines the cryptography applied to the payload, and thepayload is a JSON structure, that is encoded in JWS (Base64 encoded) or JWE (encrypted).

JWS is less secure since it is only a Base64 encoded JSON, and this means it is not suitable in our case. Because the token will carry authentication data for access to the resource and Base64 encoding is different to encryption – it is relatively trivial to decode.

JWE is more suitable to our case, because the JSON payload is encrypted (as defined in the header), and the JWT token keys are stored securely on the on the ApiOpenStudio server, so only the API server can decrypt the body.

There are multiple encryption standards available, and in our case we are using the fantastic lcobucci-jwt library, which is sponsored by one of the leading Authorisation services: Auth0. This provides support for many, many symmetric and asymmetric algorithms.

We have not implemented symmetric algorithms (these are less secure and are the same key for encryption and decryption), so with asymmetric algorithms, the public key can be used by any clients, and the public/private keys are stored in a secure location on the ApiOpenStudio server.

JWT token structure

In its compact form, JSON Web Tokens consist of three parts separated by dots (.), which are:

  • Header
  • Payload
  • Signature

Therefore, a JWT typically looks like the following.

xxxxx.yyyyy.zzzzz

Header

The header typically consists of two parts: the type of the token, which is JWT, and the signing algorithm being used, such as HMAC SHA256 or RSA.

For example:

{
  "alg": "RSA256",
  "typ": "JWT"
}

Then, this JSON is Base64Url encoded to form the first part of the JWT.

Payload

The second part of the token is the payload, which contains the claims. There are three types of claims: registeredpublic, and private claims.

  • Registered claims: These are a set of predefined claims which are not mandatory but recommended, to provide a set of useful, interoperable claims. Some of them are: iss (issuer), exp (expiration time), sub (subject), aud (audience). Notice that the claim names are only three characters long as JWT is meant to be compact.
  • Public claims: These can be defined at will by those using JWTs. But to avoid collisions they should be defined in the IANA JSON Web Token Registry or be defined as a URI that contains a collision resistant namespace.
  • Private claims: These are the custom claims created to share information between parties that agree on using them and are neither registered or public claims.

An example payload could be:

{
  "iss": "my.apiopenstudio.com",
  "sub": "1234567890",
  "name": "John Dory",
  "admin": true
}

The payload is then Base64Url encoded to form the second part of the JSON Web Token.

Signature

To create the signature part you have to take the encoded header, the encoded payload, a secret, the algorithm specified in the header, and sign that.

The signature is used to verify the message wasn’t changed along the way, and in the case of tokens signed with a private key, it can also verify that the sender of the JWT is who it says it is.

Using JWT tokens

The finalxxxxx.yyyyy.zzzzz token is sent in the request as a bearer token in the request header, e.g.:

Authorization: Bearer <token>

When the request is received by the API server, it will first confirm that the token is valid, checking it can be decrypted, the issuer, expiry date and mandatory claims.

If this all passes, the processing can continue, otherwise a 401 error response is sent and the client will need to generate a new token, using the provided core token request (auth/token) and resend the request with the token that was received.

From a processes POV, this is exactly the same as before. However, as mentioned, we are using asymmetric encoding, so we can include data in the token payload that means that ApiOpenStudio does not need too fetch user data in order to validate the user’s permissions against the resource. That only needs to be fetched once – when the token is generated.

Minor issue (caveat)

The original authorisation tokens were stateful, i.e. the token was stored against the user, along with the TTL for the token. This meant that if a user was banned, deleted or made inactive, they would instantly not be able to make any further API requests, due to either the users not being present/active anymore, or the token was no longer valid.

JWT tokens are stateless. This means that if a user has a valid token, they can still use it until it expires regardless of whether they are made inactive or deleted (they would only be prevented from fetching a fresh token).

This can be mitigated by setting a global jwt_life of less than 1 hour (the default), however this needs to be balanced against the increase in requests, since a lower token TTL will lead to more requests due to tokens frequently passing their expiry date more frequently.

Another mitigation for extreme cases can be to add the IP address of the client to the blacklist – this will prevent all future calls from that location, immediately.

3rd party authorisation integration

Because the token requires certain client data to be present, the user details and roles will need to be accessible by the authorisation provider, so that they can generate a valid body. Thankfully, most reputable providers allow you to upload these details to your account with them, and also provide ways to ensure that these details are always current and up to date.

You will need to ensure that the following mandatory claims:

  • iss – JWT issuer (your auth provider)
  • aud – permitted for (your api)
  • iat – JWT issued time
  • exp – JWT expiry time

The following ctsom claims are also included

  • uid – user ID
  • roles – complete list of roles and accounts/applications that the user is associated with

The roles object

This is in the JSON object format of:

roles: [
    {
        "role_name": <role_machine_name>,
        "accid": <account_id>,
        "appid": <application_id>
    }
]

For example:

roles: [
    {
        "role_name": "administrator",
        "accid": null,
        "appid": null
    },
    {
        "role_name": "consumer",
        "accid": 34,
        "appid": 5
    },
    {
        "role_name": "developer",
        "accid": 34,
        "appid": 5
    },
    ....
]

Note that:

  • “administrator” does not require accid or appid
  • “account_manager” does not require appid

Summary

We’re really excited to be implementing this technology, seeing the decrease in resource processing time, increasing the security of the API, and making it even easier for enterprise scale users to implement ApiOpenStudio on a large scale.

I’ll be honest with you too, it was really good fun to implement, and we totally got a total nerd-on, doing the research and coding for this!

Increased security and speed with JWT tokens

Current dev work is almost complete for implementing authorisation with JWT token for all resources! This will be part of the upcoming Beta release.

The ticket can be viewed in Gitlab.

This will replace the existing alpha version of a custom token and token TTL for each user in the user table.

It is quite important to note, before we move on, that JWT tokens are a different thing to oauth2, implicit grant, explicit grant, application grant and PKSE authorisation flow. JWT is only a standard for tokens. If you need to implement oauth2 or other similar workflows this is separate from the JWT implementation.

The problem

The problem with the former approach, was that resource requests had to make DB calls to the user, user_roles, roles, account and application tables in order to verify user permissions to that particular resource, FOR EVERY API CALL. This obviously negatively impacted performance for API calls.

This also meant that authorisation was not easily scalable to authorisation servers for enterprise implementation, because the implementation of the token and authorisation for API calls was tightly coupled to the ApiOpenStudio database and several of its tables.

The solution

Although the former approach was stateful (it maintained login state, so users could login and out), the stateless JWT token approach means that the token does not need to be stored in the database. The downside of stateless JWT tokens, is that there is no logout state. So if a user’s access is revoked, they will still have access to resources until their current token goes stale.

However, this can be mitigated by making the JWT token lifetime short in the ApiOpenStudio configuration.

Each JWT token contains custom claims for user ID and all roles that that user has. So when the initial request is received by ApiOpenStudio, it just decrypts the token and validates the user’s roles against the resources account/application and permissible user roles (i.e. Does the current user have the required role access to the account & application?).

Knock-on effects

The following processors have been retired:

  • user_login.
  • user_logout.

Nearly all core resources have been updated use the new processors:

  • generate_token (generate a valid JWT token for a user, with custom claims: uid, user roles).
  • validate_token (validate the Authorization token as a valild JWT token).
  • validate_token_roles ((validate the Authorization token as a valild JWT token and also validation the user has the correct role permissions for the resource).
  • bearer_token (not used by core atm, but preserved for any processors that need access to the bearer token).

Processors have been optimised, now that they do not need to do any pre-validation on who can do what – this is left to the core resource definitions.

Tests are updated to incorporate the changes, and also now have multiple test users with different roles.

The good news

Not only has this significantly improved the API response time, it has now made the API much more scalable for enterprise. We communicated and researched several major 3rd party authorisation services, including auth0, to make sure that the decision to move to JWT tokens and custom claims would still be viable if a 3rd party auth server was used.

Most 3rd party authorisation services implement linking into external databases, so that would take the heat off the api server for token generation, and allow the token generation to be completely decoupled from ApiOpenStudio. This will be the subject of a future post.

WP Twitter Auto Publish Powered By : XYZScripts.com