Staff Senior Systems Engineer, OT

Your Impact at LILA

Lila Sciences is seeking a Staff Senior Systems Engineer, OT to design, build, commission, and maintain the  compute, storage, network, and virtualization infrastructure that underpins Lila’s Operational Technology environment across instruments, automation platforms, lab compute, and building automation systems. This role owns hands-on systems engineering for OT and Lab IT infrastructure and turns OT network and IT/OT convergence architecture into commissioned, reliable, secure systems that scientists and automation depend on every day.

This is a senior individual contributor role reporting to the Senior Director, OT Operations & Security, working in close partnership with OT security architecture, corporate IT, controls and automation, the SOC, and laboratory operations leadership. Architectural intent — network topology, segmentation and conduit boundaries, storage tiering, host build standards — is set collaboratively; this role owns taking that intent from design into commissioned, validated, documented systems and keeping them running. The right person operates with high autonomy inside an established architecture, sets technical direction for implementation patterns, engineering standards, and operational practice, and serves as the senior technical authority for how OT infrastructure is built, commissioned, and maintained across Lila’s OT estate.

This is a foundational hire for Lila’s OT program. Lila is building autonomous laboratories where reliability and security are designed into laboratory platforms before they ship, and our governing survivability standard is simple: if the WAN drops for 72 hours, science continues — on-premises first, locally survivable, with cloud as an extension rather than a dependency. The work spans new buildouts and standardization across active lab sites, so bringing existing environments up to a common standard is as much of this role as building what comes next.

What You'll Be Building

•  Design, administer, commission, and maintain on-premises compute, storage, and network infrastructure supporting OT and Lab IT — from instrument-attached PCs and edge compute to lab orchestration servers and lab-area network gear across multiple sites.
• Build and operate virtualized environments (Proxmox VE today; VMware vSphere and Hyper-V experience transfers), including host provisioning, resource allocation, high availability, clustering, patching, and lifecycle management.
•  Implement and validate OT network configurations — VLANs, routing, firewall policy, and zone-and-conduit segmentation aligned to IEC 62443 — within the established addressing standard and topology, and understand how on-premises OT connects to SD-WAN and cloud services without exposing scientific IP.
• Own commissioning and technical sign-off for new automation platforms, instrument deployments, and lab buildouts: rack-and-stack, network cutover, golden-image and SKU standardization, validation, and post- cutover stabilization. Instruments integrate through serial-to-Ethernet and USB-to-Ethernet device servers rather than direct host passthrough, keeping virtual machines host-agnostic and migration-viable.
• Operate and extend a signed, Git-backed provisioning model with hardware-rooted host identity, and build the infrastructure-as-code and automation (Terraform, PowerShell, Python, APIs) that makes provisioning,configuration, and lifecycle management repeatable and auditable.
• Maintain the instrument host golden image standard on Windows IoT Enterprise LTSC across defined build classes, including application allowlisting, kernel-mode code integrity and Secure Boot posture, and documented exceptions where vendor software constrains standard hardening.
• Administer core infrastructure services for OT and Lab IT — Active Directory, Group Policy, DNS, DHCP, failover clustering, and certificate and machine identity — and own IP address management and addressing standards for the OT estate as it scales.
•  Design and operate backup, disaster recovery, and business continuity for OT-critical systems, holding the distinction between layers: image-level backup on dedicated hardware in its own fault domain, pull-based device configuration backup for OT endpoints, a self-hosted Git forge as source of record, and tiered storage from hot NVMe through NAS file services to object-based cold archive. 
• Deploy, tune, and operate OT visibility and asset platforms, and maintain asset inventory as the system of record for the OT estate — sensor coverage, enrichment, inventory hygiene, and the tooling that keeps it accurate.
•  Lead patch strategy, vulnerability remediation, and lifecycle and end-of-life planning within OT change-control windows, and specify and source hardware under real constraints: TAA compliance with no adversary-nation ownership or origin is a hard requirement, written vendor attestation is required per quote, and enterprise lead times are a planning input.
• Partner with OT security engineering on segmentation enforcement, machine identity, secure vendor and remote access, and monitoring and sensor coverage.
• Serve as the senior technical escalation point for complex compute, storage, network, and virtualization issues across the lab perimeter, including root-cause and problem management, within the defined IT/OT service ownership boundary.
• Produce runbooks, engineering standards, SOPs, and as-built documentation as engineering deliverables others execute from; direct and review contracted engineering and managed-service partners under Lila change control; and mentor engineers as the OT function scales.

What You'll Need to Succeed

• Significant hands-on experience designing, deploying, commissioning, and operating enterprise compute, storage, and network infrastructure — in OT, laboratory, industrial control systems, manufacturing, infrastructure, or similarly operationally constrained environments.
• Hands-on virtualization experience — design, deploy, and operate. Proxmox VE is our platform; deep VMware vSphere or Hyper-V experience transfers well.
•  Deep systems administration across Windows Server and Windows client, and Linux where applicable: Active Directory, Group Policy, DNS, DHCP, and failover clustering.
•  Strong networking and segmentation fundamentals: VLANs, firewall policy, TCP/IP, routing, DNS, DHCP, and packet analysis, plus a working understanding of how on-premises infrastructure connects to cloud.
•  Experience taking infrastructure from procurement and design through cutover and stabilization — commissioning systems end to end, not just configuring them in isolation.
• Infrastructure-as-code and scripting experience (Terraform, PowerShell, Python, APIs), and the instinct to automate a task rather than repeat it.
• Experience designing and operating backup, disaster recovery, and business continuity solutions where local survivability, not cloud failover, is the requirement.
• Comfort working in a brownfield estate — inherited vendor layouts, fixed mechanical constraints, equipment that cannot be replaced on your schedule — and the ability to execute against an architecture you did not author, with the judgment and candor to raise it when the floor contradicts the design.
•  Strong written and verbal communication skills, including the ability to explain technical decisions to engineers, scientists, security leaders, and executives, and to write documentation others can follow without you in the room.
• Willingness to work on-site at Lila laboratory locations on a regular basis, with periodic travel to other Lila sites including international locations, and participation in on-call rotation and scheduled maintenance or incident response coverage

Bonus Points For

• Experience in life sciences, biotechnology, pharmaceutical, laboratory automation, manufacturing, or high- throughput research environments.
• Familiarity with IEC 62443, NIST SP 800-82, NIST CSF, GxP, 21 CFR Part 11, ISO 9001, or comparable quality and security frameworks.
• Production experience with Windows IoT Enterprise LTSC, WDAC or AppLocker allowlisting, or Secure Boot and HVCI on constrained instrument hosts.
• Experience with OT visibility and asset platforms such as Claroty, Tenable OT, Dragos, or Nozomi Networks.
• Experience with Proxmox VE and Proxmox Backup Server, ZFS or TrueNAS, enterprise storage (SAN/NAS), hyperconverged infrastructure, or S3-compatible object storage.
• Experience with enterprise campus switching (Juniper, Cisco, or comparable), enterprise firewalls (Palo Alto or comparable), and SD-WAN or SASE platforms.
• Cloud infrastructure experience (Azure, AWS, or GCP) and hybrid on-prem-to-cloud connectivity patterns.
• Experience with PKI, certificate lifecycle management, TLS/mTLS, TPM-bound credentials, or modern machine-identity patterns.
• Experience with device configuration backup tooling (Octoplant or comparable), self-hosted Git forges, IPAM (NetBox or comparable), or ITSM platforms (Freshservice or comparable).
• Relevant certifications such as Microsoft (Windows Server Hybrid Administrator, MCSE), VMware VCP, HashiCorp Terraform Associate, Azure or AWS, Cisco (CCNP), Juniper, GICSP, IEC 62443 Cybersecurity Expert, or CISSP