Safe your Amazon Kendra indexes with the ACL utilizing a JWT shared secret key

Globally, many organizations have crucial enterprise knowledge dispersed amongst numerous content material repositories, making it tough to entry this data in a streamlined and cohesive method. Making a unified and safe search expertise is a major problem for organizations as a result of every repository incorporates a variety of doc codecs and entry management mechanisms.

Amazon Kendra is an clever enterprise search service that permits customers to look throughout completely different content material repositories. Prospects are answerable for authenticating and authorizing customers to realize entry to their search software, and Amazon Kendra allows safe seek for enterprise functions, ensuring that the outcomes of a consumer’s search question solely embrace paperwork the consumer is permitted to learn. Amazon Kendra can simply validate the id of particular person customers in addition to consumer teams who carry out searches with the addition of safe search tokens. By including consumer tokens for safe search, performing access-based filtered searches in Amazon Kendra is simplified and secured. You possibly can securely cross consumer entry data within the question payload as a substitute of utilizing attribute filters to perform this. With this characteristic, Amazon Kendra can validate the token data and robotically apply it to the search outcomes for correct and safe access-based filtering.

Amazon Kendra helps token-based consumer entry management utilizing the next token sorts:

• Open ID
• JWT with a shared secret
• JWT with a public key
• JSON

Beforehand, we noticed an indication of token-based user access control in Amazon Kendra with Open ID. On this put up, we show token-based consumer entry management in Amazon Kendra with JWT with a shared secret. JWT, or JSON Internet Token, is an open commonplace used to share safety data between a shopper and a server. It incorporates encoded JSON objects, together with a set of claims. JWTs are signed utilizing a cryptographic algorithm to make sure that the claims can’t be altered after the token is issued. JWTs are helpful in eventualities concerning authorization and data trade.

JWTs encompass three components separated by dots (.):

• Header – It incorporates components like sort of the token, which is JWT, the signing algorithm getting used, akin to HMAC SHA256 or RSA, and an elective key identifier.
• Payload – This incorporates a number of key-value pairs, known as claims, that are issued by the id supplier. Along with a number of claims referring to the issuance and expiration of the token, the token also can comprise details about the person principal and tenant.
• Signature – To create the signature half, you’re taking the encoded header, the encoded payload, a secret, the algorithm specified within the header, and signal that.

Subsequently, a JWT appears like the next:

The next is a pattern header:

{
  "alg": "HS256",
  "typ": "JWT",
  "child": "jwttest"
}

The next is the pattern payload:

{
  "teams": [
    "AWS-SA"
  ],
  "username": "John",
  "iat": 1676078851,
  "exp": 1676079151,
  "iss": "0oa5yce4g2sQdftHV5d7",
  "sub": "0oa5yce4g2sQdftHV5d7",
  "jti": "e3d62304-6608-4b72-ac0a-cb1d7049df5b"
}

The JWT is created with a secret key, and that secret secret’s personal to you, which suggests you’ll by no means reveal that to the general public or inject it contained in the JWT. While you obtain a JWT from the shopper, you possibly can confirm the JWT with the key key saved on the server. Any modification to the JWT will end in verification (JWT validation) failure.

This put up demonstrates the pattern use of a JWT utilizing a shared entry key and its utilization to safe Amazon Kendra indexes with entry controls. In manufacturing, you employ a safe authentication service supplier of your selection and primarily based in your necessities to generate JWTs.

To be taught extra about JWTs, seek advice from Introduction to JSON Web Tokens.

Answer overview

Just like the post with Open ID, this answer is designed for a set of customers and teams to make search queries to a doc repository, and outcomes are returned solely from these paperwork which might be approved for entry inside that group. The next desk outlines which paperwork every consumer is permitted to entry for our use case. The paperwork getting used on this instance are a subset of AWS public paperwork.

The next diagram illustrates the creation of a JWT with a shared entry key to regulate entry to customers to the precise paperwork within the Amazon Kendra index.

When an Amazon Kendra index receives a question API name with a consumer entry token, it validates the token utilizing a shared secret key (saved securely in AWS Secrets Manager) and will get parameters akin to username and teams within the payload. The Amazon Kendra index filters the search outcomes primarily based on the saved Entry Management Record (ACL) and the data obtained within the consumer’s JWT. These filtered outcomes are returned in response to the question API name made by the applying.

Conditions

With the intention to observe the steps on this put up, be sure to have the next:

Generate a JWT with a shared secret key

The next pattern Java code reveals find out how to create a JWT with a shared secret key utilizing the open-source jsonwebtoken bundle. In manufacturing, you’ll be utilizing a safe authentication service supplier of your selection and primarily based in your necessities to generate JWTs.

We cross the username and teams data as claims within the payload, signal the JWT with the shared secret, and generate a JWT particular for that consumer. Present a 256 bit string as your secret and retain the worth of the base64 URL encoded shared secret to make use of in a later step.

import javax.crypto.SecretKey;
import java.nio.charset.StandardCharsets;
import java.time.Prompt;
import java.time.temporal.ChronoUnit;
import java.util.*;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.safety.Keys;

//HS256 token technology

public class TokenGeneration {
    public static void most important(String[] args) {
        String secret = "${yourSecret}";
        String base64secret = Base64.getUrlEncoder().encodeToString(secret.getBytes(StandardCharsets.UTF_8));
        System.out.println("base64secret  " + base64secret);
        SecretKey sharedSecret = Keys.hmacShaKeyFor(secret.getBytes(StandardCharsets.UTF_8));
        Prompt now = Prompt.now();
        String sub = "${yourSub}";
        String username = " ${yourUsername}";
        Record<String> teams = Arrays.asList("${yourGroup}");
        //set claims
        Map<String, Object> claims = new HashMap<>();
        claims.put("username",
                username);
        claims.put("teams",
                teams);
  Map<String, Object> headers = new HashMap<String, Object>();
  headers.put("typ", "JWT");
  headers.put("child","${child}");

        String jwt = Jwts.builder()
                .setClaims(claims)
                .setIssuedAt(Date.from(now))
                .setExpiration(Date.from(now.plus(5L, ChronoUnit.MINUTES)))
                .setIssuer(sub)
                .setSubject(sub)
                .setId(UUID.randomUUID().toString())
                .setHeaderParams(headers)
                .signWith(sharedSecret)
                .compact();
        System.out.println(“JWT Token: “ + jwt);
    }
}

Create an Amazon Kendra index with a JWT shared secret

For directions on creating an Amazon Kendra index, seek advice from Creating an index. Notice down the AWS Identity and Access Management (IAM) function that you just created through the course of. Present the function entry to the S3 bucket and Secrets and techniques Supervisor following the principle of least privilege. For instance insurance policies, seek advice from Example IAM identity-based policies. After you create the index, your Amazon Kendra console ought to appear to be the next screenshot.

Full the next steps so as to add your secret:

1. On the Amazon Kendra console, navigate to the Consumer entry management tab in your index element web page.
2. Select Edit settings.

3. As a result of we’re implementing token-based entry management, choose Sure beneath Entry management settings.

4. Underneath Token configuration, select JWT with shared secret for Token sort.
5. For Sort of secret, select New.
6. For Secret title, enter AmazonKendra-jwt-shared-secret or any title of your selection.
7. For Key ID, enter the important thing ID to match your JWT that you just created within the pattern Java code.
8. For Algorithm, select the HS256 algorithm.
9. For Shared secret, enter your retained base64 URL encoded secret generated from the Java code beforehand.
10. Select Save secret.

The key will now be saved in Secrets and techniques Supervisor as a JSON Internet Key Set (JWKS). You possibly can find it on the Secrets and techniques Supervisor console. For extra particulars, seek advice from Using a JSON Web Token (JWT) with a shared secret.

11. Develop the Superior configuration part.

On this step, we arrange the consumer title and teams that shall be extracted from JWT claims and matched with the ACL when the signature is legitimate.

12. For Username¸ enter username.
13. For Teams, enter teams.
14. Depart the elective fields as default.
15. Select Subsequent, then select Replace.

Put together your S3 bucket as a knowledge supply

To organize an S3 bucket as a knowledge supply, create an S3 bucket. Within the terminal with the AWS Command Line Interface (AWS CLI) or AWS CloudShell, run the next instructions to add the paperwork and metadata to the information supply bucket:

aws s3 cp s3://aws-ml-blog/artifacts/building-a-secure-search-application-with-access-controls-kendra/docs.zip .
unzip docs.zip
aws s3 cp Information/ s3://<REPLACE-WITH-NAME-OF-S3-BUCKET>/Information/ --recursive
aws s3 cp Meta/ s3://<REPLACE-WITH-NAME-OF-S3-BUCKET>/Meta/ --recursive

The paperwork being queried are saved in an S3 bucket. Every doc sort has a separate folder: blogs, case-studies, analyst-reports, user-guides, and white-papers. This folder construction is contained in a folder named Information. Metadata information together with the ACLs are in a folder named Meta.

We use the Amazon Kendra S3 connector to configure this S3 bucket as the information supply. When the information supply is synced with the Amazon Kendra index, it crawls and indexes all paperwork in addition to collects the ACLs and doc attributes from the metadata information. To be taught extra about ACLs utilizing metadata information, seek advice from Amazon S3 document metadata. For this instance, we use the customized attribute DocumentType to indicate the kind of the doc. After the add, your S3 bucket construction ought to appear to be the next screenshot.

To set the customized attribute DocumentType, full the next steps:

1. Select your Kendra index and select Side definition within the navigation pane.
2. Select Add area.

3. For Discipline title, enter DocumentType.
4. For Information sort, select String.
5. Select Add.

Now you possibly can ingest paperwork from the bucket you created to the Amazon Kendra index utilizing the S3 connector. For full directions, seek advice from Ingesting Documents through the Amazon Kendra S3 Connector.

6. Within the Configure sync settings part, for Enter the information supply location, enter your S3 bucket (s3://kendra-demo-bucket/).
7. For Metadata information prefix folder location, enter Meta/.
8. Develop Extra configuration.
9. On the Embrace patterns tab, for Prefix, enter Information/.

For extra details about supported connectors, see Connectors.

10. Select Subsequent, then Subsequent once more, then Replace.
11. Await the information supply to be created, then choose the information supply and select Sync now.

The info supply sync can take 10–quarter-hour to finish. When your sync is full, Final sync standing ought to present as Profitable.

Question an Amazon Kendra index

To run a check question in your index, full the next steps:

1. On the Amazon Kendra console, select Search listed content material within the navigation pane.
2. Develop Take a look at question with an entry token.
3. Select Apply token.

4. We are able to generate a JWT for the consumer and group. On this instance, we create a JWT for the AWS-SA group. We substitute username as Mary and teams as AWS-SA within the JWT technology step.
5. Enter the generated token and select Apply.

Based mostly on the ACL, we needs to be outcomes from all of the folders: blogs, consumer guides, case research, analyst experiences, and whitepapers.

Equally, when logged in as James from the AWS-Gross sales group and passing the corresponding JWT, we’ve got entry to solely blogs, consumer guides, and case research.

We are able to additionally search the index as a visitor with out passing a token. The visitor is just in a position to entry contents within the blogs folder.

Experiment utilizing different queries you possibly can consider whereas logged in as completely different customers and teams and observe the outcomes.

Clear up

To keep away from incurring future prices, clear up the sources you created as a part of this answer. To delete the Amazon Kendra index and S3 bucket created whereas testing the answer, seek advice from Cleanup. To delete the Secrets and techniques Supervisor secret, seek advice from Delete an AWS Secrets Manager secret.

Conclusion

On this put up, we noticed how Amazon Kendra can carry out safe searches that solely return search outcomes primarily based on consumer entry. With the addition of a JWT with a shared secret key, we are able to simply validate the id of particular person customers in addition to consumer teams who carry out searches. This comparable method could be prolonged to a JWT with a public key. To be taught extra, seek advice from Using a JSON Web Token (JWT) with a shared secret.

Concerning the Authors

Nitin Eusebius is a Sr. Enterprise Options Architect at AWS with over 18 years of expertise in Software program Engineering and Enterprise Structure. He works with prospects on serving to them construct well-architected functions on the AWS platform. He’s captivated with fixing expertise challenges and serving to prospects with their cloud journey.

Kruthi Jayasimha Rao is a Associate Options Architect with a spotlight in AI and ML. She supplies technical steering to AWS Companions in following greatest practices to construct safe, resilient, and extremely obtainable options within the AWS Cloud.

Ishaan Berry is a Software program Engineer at Amazon Internet Companies, engaged on Amazon Kendra, an enterprise search engine. He’s captivated with safety and has labored on key elements of Kendra’s Entry Management options over the previous 2 years.

Akash Bhatia is a Principal Options architect with AWS. His present focus helps enterprise prospects obtain their enterprise outcomes by architecting and implementing revolutionary and resilient options at scale. He has been working in expertise for over 15 years at firms starting from Fortune 100 to start-ups in Manufacturing, Aerospace and Retail verticals.