Allow or disable ACL crawling safely in Amazon Q Enterprise

Amazon Q Business not too long ago added assist for directors to switch the default entry management record (ACL) crawling characteristic for knowledge supply connectors.

Amazon Q Enterprise is a completely managed, AI powered assistant with enterprise-grade safety and privateness options. It consists of over 40 data source connectors that crawl and index paperwork. By default, Amazon Q Enterprise indexes ACL info connected to paperwork together with the paperwork themselves and makes use of this to filter chat responses based mostly on the person’s doc entry. With this new characteristic, you possibly can allow or disable ACL crawling as required by their enterprise use case.

This put up introduces the brand new ACL toggle characteristic for Amazon Q Enterprise, which you should use to allow or disable ACL crawling. We’ll discover use instances for disabling ACLs and focus on how you can safely allow or disable ACL crawling.

Overview of entry management record crawling

Amazon Q Enterprise knowledge supply connectors assist crawl numerous knowledge sources to gather and index content material in Amazon Q Enterprise for quick discovery and retrieval when answering person queries. These knowledge sources typically include paperwork with totally different classifications akin to public, inner public, personal, and confidential. To offer fine-grained management over entry rights, you possibly can connect ACLs to paperwork, permitting you to specify totally different ranges of entry for numerous customers or teams. To confirm that Amazon Q Enterprise respects entry management insurance policies and that customers solely obtain responses for content material they’re approved to entry, the information supply connectors robotically crawl for entry permissions related to the content material, person identifiers, and teams.

The previous determine illustrates the Amazon Q Enterprise knowledge supply crawler with ACL crawling enabled. Because the connector retrieves content material from the information supply, it examines the related ACL and compiles an inventory of customers and teams with learn permissions for every doc. The connector additionally collects person identifiers, that are saved within the Amazon Q Enterprise person retailer for fast matching throughout question execution. Each the ACL and content material are optimized and saved within the Amazon Q Enterprise index storage, enabling safe and environment friendly retrieval when answering person queries. For extra info on the person retailer, see Understanding Amazon Q Business User Store.

When to disable ACL crawling?

ACL crawling builds a security-aware index that respects entry management insurance policies within the major knowledge supply. This course of helps keep knowledge privateness and entry management required for regulatory compliance, ensuring that delicate info isn’t inadvertently uncovered by way of person question outcomes. It gives a scalable mechanism to deal with massive quantities of content material whereas sustaining consistency between the precise entry controls on the information and what’s discoverable by way of search. Due to these benefits, ACL crawling is strongly beneficial for all knowledge sources. Nonetheless, there are some circumstances while you may must disable it. The next are some explanation why you may disable ACL crawling.

Internally public content material

Organizations typically designate sure knowledge sources as internally public, together with HR insurance policies, IT information bases, and wiki pages. For example, an organization may allocate a complete Microsoft SharePoint web site for insurance policies accessible to all staff, classifying it as internal-public. In such instances, crawling ACLs for permissions that embrace all staff may be expensive and create pointless overhead. Turning off ACL crawling is perhaps advantageous in these eventualities.

Information supply comprises irreconcilable identities

Amazon Q Enterprise requires all customers to authenticate with an enterprise-approved identification supplier (IdP). After profitable authentication, Amazon Q Enterprise makes use of the IdP-provided person identifier to match in opposition to the person identifier fetched from the information supply throughout ACL crawling. This course of validates person entry to content material earlier than retrieving it for question responses.

Nonetheless, due to legacy points akin to mergers and acquisitions, knowledge supply configuration limitations, or different constraints, the first person identifier from the IdP may differ from the one within the knowledge supply. This discrepancy can stop Amazon Q Enterprise from retrieving related content material from the index and answering person queries successfully.

In such instances, it is perhaps essential to disable ACL crawling and use various choices. These embrace implementing attribute filters or constructing devoted restricted functions with entry restricted to particular audiences and content material. For extra info on attribute filters, see Filtering chat responses using document attributes.

Use case-driven focused deployments

As a completely managed service, Amazon Q Enterprise may be rapidly deployed in a number of cases for scoped down focused use instances. Examples embrace an HR bot in Slack or an AI assistant for buyer assist brokers in a contact heart. As a result of these AI assistants is perhaps deployed for a restricted viewers, and the listed content material is perhaps usually out there to all customers with utility entry, ACL crawling may be turned off.

Be aware of warning

Amazon Q Enterprise can’t implement document-level entry controls if ACL crawling is disabled. When ACL crawling is disabled for a knowledge supply, listed content material in that supply shall be thought-about accessible to customers with entry to the Amazon Q Enterprise utility. Due to this fact, disabling ACL crawling must be executed with warning and due diligence. The next are some beneficial greatest practices:

Notify knowledge supply content material house owners and directors of your intent to disable ACL crawling and procure their approval beforehand.
If relevant, contemplate implementing various choices akin to attribute filtering to limit content material retrieval or deploying a scoped-down, use-case-driven deployment to a restricted viewers.
Preserve a choice doc that clearly articulates the explanations for disabling ACL crawling, the scope of affected content material, and precautions taken to stop indexing of delicate info.

Be aware: As a precaution, you can not disable ACL crawling for an current Amazon Q Enterprise knowledge supply that already has ACL crawling enabled. To disable ACL crawling, you could delete the information supply and recreate it. You’ll be able to solely disable ACL crawling throughout the knowledge supply creation course of, and this requires an account administrator to grant permission for disabling ACL crawling when configuring the information supply.

Procedures for configuring ACL crawling

Amazon Q Enterprise ACL crawling helps defend your knowledge. Amazon Q Enterprise gives safeguards to assist directors and builders mitigate by accident disabling ACL crawling. On this part, we are going to cowl how one can permit or deny the ACL crawling disable characteristic, discover procedures to allow or disable ACL crawling, clarify how you can monitor logs for ACL crawling configuration modifications, and troubleshoot widespread points.

Personas for configuring ACL crawling

ACL crawling configuration usually includes a number of roles, relying in your organizational construction. To maximise safeguards, it’s beneficial that these roles are stuffed by totally different people. For sooner deployments, determine the required personnel inside your group earlier than beginning the venture and guarantee they collaborate to finish the configuration. Listed below are the widespread roles wanted for ACL crawling configuration:

AWS account administrator – An AWS account administrator is a person with full entry to AWS companies and the power to handle IAM sources and permissions within the account. They will create and handle organizations, enabling centralized administration of a number of AWS accounts.
Amazon Q Enterprise administrator – An Amazon Q Enterprise administrator is often a person or function liable for managing and configuring the Amazon Q Enterprise service. Their duties embrace creating and optimizing Amazon Q Enterprise indexes, organising guardrails, and tuning relevance. Additionally they arrange and keep connections to varied knowledge sources that Amazon Q Enterprise will index, akin to Amazon Simple Storage Service (Amazon S3) buckets, SharePoint, Salesforce, and Confluence.

Stipulations for ACL crawling

Amazon Q Enterprise utility.
Amazon Q Enterprise knowledge supply connector that helps ACL crawling configuration.
Information supply authentication that meets the permissions required for crawling content material and ACLs.

Course of to disallow the choice to disable ACL crawling

By default, the choice to disable ACL crawling is enabled for an account. AWS account directors can disallow this characteristic by organising an account-level coverage. It’s beneficial to configure an express deny for manufacturing accounts by default. The next beneath reveals the related actions in relation to the personas concerned within the configuration course of.

Directors can connect the IAM motion qbusiness:DisableAclOnDataSource to the Amazon Q Enterprise administrator person or function coverage to disclaim or permit the choice to disable ACL crawling. The instance IAM coverage code snippet that follows demonstrates how you can arrange an express deny.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
          "Effect": "Deny",
          "Action": [
                "qbusiness:DisableAclOnDataSource"
            ],
          "Useful resource": ["*"]
       }
    ]
}

Be aware that even when the choice to disable ACL crawling is denied, the person interface may not grey out this feature. Nonetheless, should you try and create a knowledge supply with this feature disabled, it’s going to fail the validation verify, and Amazon Q Enterprise is not going to create the information supply.

Course of to disable ACL crawling for a knowledge supply connector

Earlier than organising a knowledge supply connector with ACL crawling disabled in your Amazon Q Enterprise utility deployment, just be sure you don’t have any delicate content material within the knowledge supply or have applied controls to assist stop unintentional content material publicity. Confirm that the information supply connector helps the choice to disable ACL crawling. Notify info custodians, content material house owners, and knowledge supply directors of your intent to disable ACL crawling and procure their documented approvals, if crucial. In case your account administrator has explicitly denied the choice to disable ACL crawling, request non permanent permission. After you could have secured all approvals and exceptions, create a brand new knowledge supply with ACL crawling disabled and sync the information. With ACL crawling disabled, Amazon Q Enterprise customers will be capable of uncover information and procure solutions from the listed paperwork by way of this connector. Notify the account administrator to revert the account coverage again to explicitly denying the disable ACL crawling possibility. The method and interplay between totally different roles are proven within the following chart.

The next is an summary of the process to create a knowledge supply with ACL crawling disabled utilizing AWS Console:

Navigate to the Amazon Q Business console.
Choose the Amazon Q Enterprise utility that you simply wish to add a knowledge supply connector to.
Select Add knowledge supply within the Information sources part and choose the specified connector.
Replace the connector configuration info. See Connecting Amazon Q Business data sources for configuration particulars.
Within the Authorization part, select Disable ACLs and verify the acknowledgment to just accept the dangers of disabling ACL crawling.
Full the remaining connector configuration and select Save.
Sync the information supply.

Be aware: You can not disable ACL crawling for an current knowledge supply connector that was created with ACL crawling enabled. You could create a brand new knowledge supply connector occasion with ACL disabled and delete the older occasion that has ACL crawling enabled.

Course of to allow ACL crawling for a knowledge supply connector

Creating a knowledge supply connector with ACL crawling enabled is beneficial and doesn’t require extra permit itemizing from AWS account directors. To allow ACL crawling, you observe steps just like disabling ACLs as described within the earlier part. When configuring the information supply connector utilizing the console, select Allow ACLs within the Authorization part to create a connector with ACL crawling enabled. It’s also possible to allow ACL crawling at any time for an current knowledge supply connector that was created with this feature disabled. Sync the information supply connector for the ACL enforcement to take impact. Amazon Q Enterprise customers can solely question and procure solutions from paperwork to which they’ve entry within the authentic knowledge supply.

It’s necessary to overview that the information supply administrator has arrange the required permissions correctly, ensuring that the crawler has permission to crawl for ACLs within the knowledge supply earlier than enabling ACL crawling. You will discover the required permissions within the prerequisite part of the connector in Connecting Amazon Q Business data sources. The next reveals the method for organising a knowledge supply connector with ACL crawling enabled.

Logging and monitoring the ACL crawling configuration

Amazon Q Enterprise makes use of AWS CloudTrail for logging API calls associated to ACL crawling configuration. You’ll be able to monitor the CloudTrail log for CreateDataSource and UpdateDataSource API calls to determine ACL crawling-related modifications made to knowledge supply configuration. For an entire record of Amazon Q Enterprise APIs which might be logged to CloudTrail, see Logging Amazon Q Business API calls using AWS CloudTrail.

Directors can configure Amazon CloudWatch alarms to generate automated alert notifications if ACL crawling is disabled for a knowledge supply connector, permitting them to provoke corrective motion. For step-by-step directions on organising CloudWatch alarms based mostly on CloudTrail occasions, see How do I use CloudWatch alarms to monitor CloudTrail events.

The instance CloudWatch alarm code snippet that follows reveals the filter sample for figuring out occasions associated to disabling ACL crawling in a knowledge supply connector.

Ideas for troubleshooting

When configuring Amazon Q Enterprise knowledge supply connectors, you may sometimes encounter points. The next are some widespread errors and their doable resolutions.

Not approved to disable ACL crawling

When creating a brand new knowledge supply connector with ACL crawling disabled, you may see an error message stating not approved to carry out: qbusiness:DisableAclOnDataSource as proven within the following picture.

This error signifies that your administrator has explicitly denied the choice to disable ACL crawling to your AWS account. Contact your administrator to allow-list this motion to your account. For extra particulars, see the Course of to disable ACL crawling for a knowledge supply connector part earlier on this put up.

Information supply connection errors

Information supply connectors may additionally fail to hook up with your knowledge supply or crawl knowledge. In such instances, confirm that Amazon Q Enterprise can attain the information supply by way of the general public web or by way of a VPC personal community. See Connecting Amazon Q Business data sources to be sure that your knowledge supply authentication has the permissions wanted to crawl content material and ACLs, if enabled.

Identification and ACL mismatch errors

Lastly, after efficiently syncing knowledge with ACL crawling enabled, some customers may nonetheless be unable to get solutions to queries, although the related paperwork have been listed. This problem generally happens when the person lacks entry to the listed content material within the authentic knowledge supply, or when the person identification obtained from the information supply doesn’t match the sign-in identification. To troubleshoot such ACL mismatch points, look at the information supply sync report. For extra info, see Introducing document-level sync reports: Enhanced data sync visibility in Amazon Q Business.

Key issues and proposals

Given the influence that disabling ACL crawling can have on content material safety, contemplate these restrictions and greatest practices when disabling ACL crawling in Amazon Q Enterprise knowledge supply connectors:

ACL crawling enablement is a one-way management mechanism. After it’s enabled, you can not disable it. This helps stop by accident disabling ACL crawling in manufacturing environments.
Preserve ACL crawling enabled by default and disable it just for the subset of information supply connectors that require it.
If crucial, contemplate splitting the indexing of a knowledge supply by organising a number of knowledge supply connectors and limiting ACL crawling disablement to a smaller content material phase. Use the doc Inclusion and Exclusion characteristic of information supply connectors to outline the indexing scope.
When ACL crawling is disabled due to irreconcilable identities, contemplate various choices. These embrace implementing attribute filters, proscribing entry to the Amazon Q Enterprise utility, and setting up guardrails.
As a safety greatest apply, AWS Organizations and account directors ought to add a service management coverage to explicitly deny the qbusiness:DisableAclOnDataSource permission for all accounts. Grant this permission solely when requested by an Amazon Q Enterprise administrator. After configuring a knowledge supply connector with ACL crawling disabled, revert to an express deny. Use a ticketing system to keep up a report of exception approvals. For extra info, see Grant permission to create data sources with ACLs disabled.
At present, disabling ACL crawling is on the market for restricted connectors, together with ServiceNow, Confluence, SharePoint, Jira, Google Drive, OneDrive, Salesforce, Zendesk, GitHub, MS Groups, and Slack. For the most recent record of connectors that assist disabling ACL crawling, see Connecting Amazon Q Business data sources.

Clear up

To keep away from incurring extra expenses, ensure you delete any sources created on this put up.

To delete any knowledge supply created in Amazon Q Enterprise, observe the directions in Deleting an Amazon Q Business data source connector to delete the identical.
To delete any Amazon Q Enterprise utility created, observe the directions in Deleting an application.

Conclusion

Amazon Q Enterprise knowledge supply connector ACL crawling is an important characteristic that helps organizations construct, handle, and scale safe AI assistants. It performs an important function in implementing regulatory and compliance insurance policies and defending delicate content material. With the introduction of a self-service characteristic to disable ACL crawling, Amazon Q Enterprise now gives you extra autonomy to decide on deployment choices that fit your group’s enterprise wants. To begin constructing safe AI assistants with Amazon Q Enterprise, discover the Getting started information.

Concerning the Authors

Rajesh Kumar Ravi, a Senior Options Architect at Amazon Internet Companies, focuses on constructing generative AI options utilizing Amazon Q Enterprise, Amazon Bedrock, and Amazon Kendra. He helps companies worldwide implement these applied sciences to boost effectivity, innovation, and competitiveness. An completed expertise chief, Rajesh has expertise creating progressive AI merchandise, nurturing the builder group, and contributing to new concepts. Exterior of labor, he enjoys strolling and brief mountaineering journeys.

Meenakshisundaram Thandavarayan works for AWS as an AI/ML Specialist. He has a ardour to design, create, and promote human-centered knowledge and analytics experiences. Meena focuses on creating sustainable programs that ship measurable, aggressive benefits for strategic clients of AWS. Meena is a connector and design thinker and strives to drive enterprise to new methods of working by way of innovation, incubation, and democratization.

Amit Choudhary is a Product Supervisor for Amazon Q Enterprise connectors. He likes to construct merchandise that make it straightforward for purchasers to make use of privacy-preserving applied sciences (PETs) akin to differential privateness

Keerthi Kumar Kallur is a Software program Improvement Engineer at AWS. He’s a part of the Amazon Q Enterprise group and labored on numerous options with clients. In his spare time, he likes to do out of doors actions akin to mountaineering and sports activities akin to volleyball.

Allow or disable ACL crawling safely in Amazon Q Enterprise