Architecting TrueLook’s AI-powered development security system on Amazon SageMaker AI

This put up is co-written by TrueLook and AWS.

TrueLook is a development digicam and jobsite intelligence firm that gives real-time visibility into development tasks. Its platform combines high-resolution time-lapse cameras, stay video streaming, and AI-powered insights to assist groups monitor progress, enhance accountability, and scale back danger throughout the whole mission lifecycle.

TrueLook used Amazon SageMaker AI to construct and deploy an AI-powered development security monitoring system that robotically detects private protecting tools (PPE) by combining TrueLook’s expertise in jobsite digicam techniques utilizing the AWS machine studying (ML) infrastructure. TrueLook has constructed an answer that identifies issues of safety via automated picture evaluation to establish PPEs equivalent to exhausting hats, high-visibility security vests, security helmets, gloves, protecting eyewear, and way more. By this method, mission groups can floor unsafe working circumstances, non-compliant habits, and publicity to high-risk zones sooner, strengthening total security governance. AI helps TrueLook transfer from guide checks to a better, extra scalable method to jobsite security.

This put up gives an in depth architectural overview of how TrueLook constructed its AI-powered security monitoring system utilizing SageMaker AI, highlighting key technical selections, pipeline design patterns, and MLOps finest practices. You’ll acquire priceless insights into designing scalable pc imaginative and prescient options on AWS, significantly round mannequin coaching workflows, automated pipeline creation, and manufacturing deployment methods for real-time inference.

Building security: The important problem

Building websites are among the many most hazardous work environments, with employees dealing with dangers from heavy equipment, elevated work areas, electrical hazards, and chemical exposures. The Occupational Security and Well being Administration (OSHA) stories that development accounts for one in five worker deaths within the U.S. annually, regardless of representing a considerably smaller share of the full workforce. Past the human value, security incidents create vital monetary burdens via employees’ compensation claims, mission delays, regulatory fines, and potential litigation.

Conventional security monitoring depends closely on guide oversight with security managers conducting periodic website walks, reviewing footage after incidents happen, or relying on employees to self-report violations. Nonetheless, this method faces basic limitations:

• Scale constraints – Massive development tasks with a number of websites and a whole lot of employees can’t be successfully monitored by human observers alone
• Inconsistent protection – Guide monitoring is topic to fatigue, distraction, and human error, resulting in missed violations throughout important moments
• Reactive response – Conventional strategies usually establish issues of safety solely after incidents have occurred, limiting alternatives for prevention
• Useful resource intensive – Deploying enough human displays throughout all websites and shifts requires vital personnel funding
• Compliance gaps – Inconsistent documentation makes it troublesome to take care of the great audit trails required by OSHA and different regulatory our bodies

These challenges create a necessity for automated, scalable, security monitoring options that may present constant, real-time oversight throughout development operations.

Resolution overview

TrueLook’s AI-powered PPE detection and monitoring answer makes use of AWS infrastructure and ML to detect security compliance points in development zones via website imagery. TrueLook sources photos to make use of in PPE detection from on-site cameras. To construct, practice, and deploy these fashions, TrueLook makes use of SageMaker AI, which gives managed infrastructure for the whole ML workflow. By offloading the undifferentiated heavy lifting of infrastructure setup and orchestration to SageMaker AI, TrueLook’s group can deal with enhancing mannequin accuracy and reliability, serving to to make sure that the answer scales successfully throughout buyer websites. The next structure diagram illustrates the end-to-end workflow, highlighting how a number of AWS providers are built-in to ship a seamless, scalable AI answer.

TrueLook’s labeled picture dataset strikes via a coaching pipeline in three key levels: preprocessing (SageMaker Processing Job), coaching (SageMaker Training Job), and versioning with observability (SageMaker Model Registry). SageMaker Processing jobs dealt with picture cleansing and preparation at scale, working on single or a number of nodes relying on dataset measurement. SageMaker Coaching jobs executed the mannequin coaching utilizing built-in PyTorch containers and NVIDIA GPUs. With a primary runtime configuration with SageMaker PyTorch estimators, the identical script may run on a single-node multi-GPU setup or scale out to distributed multi-node coaching, in order that TrueLook may steadiness velocity and accuracy as wanted. Skilled fashions had been then versioned and saved within the SageMaker Mannequin Registry, offering a central hub for monitoring, governance, and deployment.

As indicated within the previous structure diagram, this workflow is orchestrated end-to-end with SageMaker Pipelines, which stitches preprocessing, coaching, and registration into an automatic, repeatable movement. By utilizing the managed MLflow integration and TensorBoard performance provided by SageMaker, TrueLook may observe experiments, examine efficiency, and supply repeatability at scale, making it simple to fine-tune fashions and ship correct PPE detection throughout its buyer development websites nationwide.

After fashions are skilled, evaluated, and accredited, deployment is dealt with via the absolutely managed internet hosting service obtainable via SageMaker AI. Actual-time endpoints ship low-latency inference at scale, powering PPE detection straight on stay video streams or snapshots. When a violation is detected, the system triggers downstream alerts that notify clients in actual time. To maintain the system constantly enhancing, TrueLook extends this pipeline with an lively studying loop. By dropping new batches of photos into Amazon Simple Storage Service (Amazon S3), the workflow robotically triggers fine-tuning or retraining via a steady integration and supply (CI/CD) course of. Earlier than promotion to manufacturing, every candidate mannequin passes via governance checks within the SageMaker Mannequin Registry, runtime analysis with MLflow, and visible inference validation with Tensorboard. Solely after these steps are full, are new fashions deployed, serving to to make sure reliability and consistency at scale.

Construct excessive performing pc imaginative and prescient object detection fashions with SageMaker AI

Coaching correct pc imaginative and prescient fashions begins with high-quality annotated information—a step that always turns into a bottleneck in growing AI-powered providers. For TrueLook, constructing a dependable PPE detection mannequin means making a labeled dataset that captured all main lessons of violations—folks, exhausting hats, security vests, security boots, and extra—beneath numerous circumstances equivalent to various scenes, lighting, orientations, and views. These annotations got here from TrueLook’s nationwide community of video digicam feeds throughout development websites.To speed up progress and enhance mannequin high quality, TrueLook’s engineering group partnered with the SageMaker AI go-to-market (GTM) information science group to design a excessive accuracy multi-stage coaching pipeline. This method lowered the time required to maneuver from experimentation to manufacturing by pairing the deep pc imaginative and prescient and information science experience of AWS and TrueLook with the simplified managed coaching and deployment workflows supported by SageMaker AI. The consequence was a scalable, multi-stage pipeline that enabled sooner iteration, simplified operational complexity, and delivered accuracy enhancements past TrueLook’s earlier state-of-the-art PPE detection fashions.

Early experiments and different approaches

TrueLook started by experimenting with different suppliers that provided UI or API pushed workflows for low-code and no-code ML and deep studying (DL) mannequin coaching. TrueLook initially used default, vendor-recommended hyperparameters and subsequently retrained fashions after adjusting uncovered parameters, equivalent to batch measurement, studying charge, and confidence thresholds, to shortly fine-tune and consider object detection fashions utilizing their very own datasets. Nonetheless, the restricted management over the coaching course of didn’t yield outcomes enough for manufacturing readiness, as a result of mannequin efficiency plateaued inside a slender vary due to the shortage of extra tuning and optimization controls. For instance, coaching with an preliminary set of 1,000 labeled photos produced imply common precision (mAP) within the 60–70% vary. Whereas this proved the feasibility of the method, outcomes additionally confirmed that efficiency scaled tightly with the variety of labeled photos obtainable, highlighting the necessity for a extra superior and scalable pipeline.

A 3-stage fine-tuning pipeline utilizing SageMaker AI

Early experimentation with low-code and no-code approaches revealed the necessity to domain-shift a pre-trained, open-domain object detection mannequin—initially skilled to acknowledge generic objects equivalent to autos, folks, and animals—into the development and security area. This preliminary area adaptation permits the mannequin to study construction-specific visible ideas, together with security tools and employee presence in complicated job-site circumstances equivalent to partial occlusion, utilizing curated open-source development datasets. This domain-shifted object detection mannequin is then additional fine-tuned on customer-specific datasets to align the mannequin’s goal lessons with every buyer’s labeling requirements and website circumstances. The next diagram illustrates this development as a three-stage coaching workflow.

1. Pretrained mannequin: Choose a pretrained Pc Imaginative and prescient (CV) object detection mannequin skilled on large-scale open-source photos
2. Area adaptation: Effective-tune a pre-trained mannequin with brazenly obtainable development security area dataset
3. Effective tuning: Effective-tune the domain-adapted mannequin on TrueLook’s annotated dataset to quickly enhance accuracy

YOLO object detection household of fashions

Earlier than analyzing the multistage coaching workflow, we need to introduce the item detection mannequin on the coronary heart of TrueLook’s AI-powered development security system.

YOLO (You Only Look Once) is a household of real-time object detection fashions optimized for quick, single-pass detection with a powerful steadiness of accuracy and throughput, making it well-suited for dynamic environments equivalent to development jobsites. YOLOv11 advances this lineage with architectural enhancements that improve characteristic extraction, ship greater accuracy with fewer parameters, and allow sooner inference, even on constrained {hardware}, whereas additionally supporting duties like segmentation and pose estimation.

Multi-stage object detection fine-tuning workflow

On this part, we describe the end-to-end method used to pick, adapt, and fine-tune a pretrained imaginative and prescient mannequin for development website security monitoring,

• Choosing a pretrained mannequin – The group evaluated pretrained fashions based mostly on elements equivalent to measurement, accuracy, coaching metrics, class protection, and licensing. YOLOv11 was chosen as the bottom mannequin for its robust efficiency and suitability for construction-related use circumstances.
• Area adaptation – Pretrained fashions are sometimes skilled on broad lessons equivalent to vehicles, animals, or on a regular basis objects. By adapting these weights to deal with construction-specific lessons—equivalent to exhausting hats, security cones, and employees in security zones—the mannequin gained area consciousness. This adaptation used brazenly obtainable datasets like Roboflow: Building Security and benefitted from information augmentation to enhance robustness throughout views, occlusions, and lighting circumstances.
• Effective-tuning with TrueLook information – The domain-adapted mannequin was then fine-tuned on TrueLook’s proprietary, high-quality labeled dataset. As a result of the mannequin already acknowledged PPE lessons fairly properly after stage two, fine-tuning sharpened its efficiency on imagery from TrueLook’s stay development feeds. Further training-time augmentations additional improved generalization beneath real-world circumstances.

This staged method proved extremely efficient. For instance, with the identical 1,000 labeled photos, the pipeline achieved mAP scores within the 80–90% vary—an enchancment of 20 factors over the alternate supplier’s workflow. One other good thing about this design was effectivity: levels one and two wanted to be run solely as soon as, producing a reusable domain-adapted mannequin. Each time new information turned obtainable, TrueLook may rerun stage three, lowering coaching time whereas constantly enhancing on total mannequin accuracy.In distinction, low-code and no-code options sometimes supply restricted management over mannequin structure, coaching technique, and multi-stage optimization, making it troublesome to carry out specific area adaptation and iterative fine-tuning at scale. Whereas these instruments can speed up preliminary prototyping, they usually fall brief when greater accuracy, reproducibility, and production-grade customization are required for complicated, real-world environments like development jobsites.

Operationalizing with SageMaker AI

By utilizing SageMaker AI, TrueLook operationalized its multi-stage object detection workflow as a scalable, production-ready MLOps framework. By utilizing managed capabilities equivalent to SageMaker Pipelines and the SageMaker Model Registry, TrueLook automated the total mannequin lifecycle, from coaching and analysis to versioning and deployment, whereas sustaining robust governance and traceability. This method lowered guide orchestration, lowered operational danger, and supplied the reliability and observability required to run AI-powered security monitoring providers at scale.

Implementing end-to-end object detection utilizing SageMaker Pipelines

Constructing an correct object detection mannequin was solely step one in constructing a complete AI-powered development security system. Ongoing enchancment required speedy iteration, managed experimentation, and dependable promotion of high-quality fashions as new information turned obtainable. To allow this, TrueLook and AWS applied an automatic workflow utilizing SageMaker Pipelines that helps parallel experimentation with the flexibility so as to add automated mannequin analysis that robotically filters out underperforming fashions and advancing solely people who meet predefined efficiency thresholds, leading to sooner iteration, improved reproducibility, and a reliable path from experimentation to manufacturing.

Creating the pipeline – Outline as soon as philosophy

TrueLook applied a reusable, parameterized workflow that automates the total lifecycle of its development security object detection fashions. The workflow begins by remodeling uncooked jobsite imagery into model-ready datasets. It then trains a YOLOv11 object detection mannequin and robotically registers the skilled mannequin in a central mannequin registry for versioning and governance. Constructed-in analysis steps measure mannequin efficiency (equivalent to mAP, F1-score, and so forth) in opposition to predefined thresholds. Fashions that meet these requirements are promoted for deployment and registered as versioned artifacts in a central mannequin registry. These registered fashions will be reviewed, commented on, accredited, or rejected via auditable workflows, whereas underperforming runs are robotically stopped to forestall low-quality fashions from reaching manufacturing.

TrueLook outlined a reusable, parameterized workflow that reduces the necessity to rebuild orchestration logic for every mannequin iteration. Groups can set off repeatable runs by adjusting datasets and coaching settings equivalent to picture decision, batch measurement, studying charges, coaching period, and information augmentation methods. They’ll additionally alter compute configurations together with GPU occasion sort, variety of GPUs, and reminiscence capability. A number of runs can execute in parallel, whereas automated gating and conditional execution implement constant high quality requirements, lowering operational overhead, minimizing human error, and accelerating steady mannequin enchancment at scale.

# Core experimentation parameters
object_detection_params = ParameterString(
    title="pre_training_params", 
    default_value="epochs=1,lr0=1e-3,batch=1"
)
...
# Coaching occasion as a parameter 
training_instance_type = ParameterString(
    title="ml_instance ", 
    default_value="ml.g6e.12xlarge"
)
...
# Stage 2 mannequin hyperparams
fine_tuning_params = ParameterString(
    title="fine_tuning_params", 
    default_value="epochs=1,lr0=1e-4,batch=1"
)

Ruled experimentation and automatic deployment

Each coaching run is robotically tracked via built-in experiment administration and mannequin registration techniques that seize parameters, metrics, and mannequin artifacts in versioned histories. This creates a searchable catalog of experimental outcomes, enabling systematic comparability of various coaching methods and identification of optimum configurations for development security detection. Accepted fashions are then robotically deployed to GPU-accelerated manufacturing endpoints utilizing versioned, timestamped naming to forestall conflicts. This creates a seamless and repeatable path from experimentation to real-time deployment, enabling speedy iteration whereas sustaining robust governance and minimal guide intervention.

Abstract

This case examine highlights how the AWS–TrueLook collaboration enabled development groups to make use of managed ML providers for scalable, production-ready security monitoring whereas avoiding heavy infrastructure overhead. It demonstrates a confirmed three-stage fine-tuning method that delivers high-accuracy development security fashions even with restricted information, surpassing what is usually achievable with low-code or no-code options. This put up additionally gives sensible steering on constructing, coaching, and deploying pc imaginative and prescient fashions utilizing AWS managed providers, and emphasizes the worth of early AWS engagement for structure design and domain-specific implementation. TrueLook’s success illustrates how industry-focused AI/ML options, backed by deep area experience, can successfully automate and elevate jobsite security operations.

In regards to the authors

Steven McDowall is a know-how and product chief with in depth expertise in product technique, product administration, and software program engineering. He at present serves as Vice President of Product at TrueLook, the place he leads the event of construction-technology and real-time video options, bringing a powerful engineering basis and user-focused method to product execution.

Scott Anderson is the Director of Platform Engineering at TrueLook, the place he leads the event and scalability of the techniques that energy the corporate’s core platform. He brings over 30 years of deep technical expertise and a practical engineering mindset, with a deal with constructing dependable, maintainable infrastructure that helps long-term product progress.

Marc Ritter is a Lead Software program Engineer at TrueLook, the place he drives the design and implementation of core platform options and contributes to superior know-how initiatives. He applies a powerful engineering mindset to fixing complicated technical challenges and enhancing the efficiency and reliability of TrueLook’s options. Marc is obsessed with leveraging considerate structure and collaborative improvement to construct scalable software program techniques.

Pranav Murthy is a Senior Generative AI Information Scientist at AWS, specializing in serving to organizations innovate with Generative AI, Deep Studying, and Machine Studying on Amazon SageMaker AI. Over the previous 10+ years, he has developed and scaled superior pc imaginative and prescient (CV) and pure language processing (NLP) fashions to sort out high-impact issues—from optimizing international provide chains to enabling real-time video analytics and multilingual search. When he’s not constructing AI options, Pranav enjoys enjoying strategic video games like chess, touring to find new cultures, and mentoring aspiring AI practitioners. Yow will discover Pranav on LinkedIn.

Gaurav Singh is a Senior Buyer Options Supervisor at AWS with over 20 years of expertise in cloud transformation and IT consulting. He focuses on guiding clients via their cloud journey, serving as a trusted advisor for migration, modernization, and innovation alternatives. Gaurav gives strategic progress steering that helps clients obtain their targets whereas leveraging AWS providers to drive innovation and operational excellence. Yow will discover Gaurav on LinkedIn.

Surya Kari is a Senior Generative AI Information Scientist at AWS, specializing in growing options leveraging state-of-the-art basis fashions. He has in depth expertise working with superior language fashions together with DeepSeek-R1, the Llama household, and Qwen, specializing in their fine-tuning and optimization for particular scientific functions. His experience extends to implementing environment friendly coaching pipelines and deployment methods utilizing AWS SageMaker, enabling the scaling of basis fashions from improvement to manufacturing. He collaborates with clients to design and implement generative AI options, serving to them navigate mannequin choice, fine-tuning approaches, and deployment methods to attain optimum efficiency for his or her particular use circumstances. Yow will discover Surya on LinkedIn.