Sim-to-Real MPC Optimization for WalkOn F1: Lessons from Cybathlon 2024

Introduction

Transferring advanced Model Predictive Control (MPC) walking trajectories from simulation to a real exoskeleton presents significant technical challenges, particularly when optimizing for speed, safety, comfort, and smoothness simultaneously. This work describes several months of research and engineering effort to optimize MPC for the WalkOn F1 exoskeleton, addressing the specific requirements of the Cybathlon 2024 competition. This post provides an account of the real-world problems encountered, and the iterative, data-driven solutions that enabled our team's success at Cybathlon 2024.

---

Cybathlon 2024

---

Fundamental Challenge: Exoskeleton vs Humanoid Robot Control

Before diving into the specific sim-to-real challenges encountered with the WalkOn F1, it's crucial to understand why exoskeleton control presents fundamentally different challenges compared to humanoid robot control. The control strategies that work for autonomous bipedal robots cannot be directly applied to human-worn exoskeletons.

Humanoid Robot Control: Known Parameters, Predictable Dynamics

Characteristics of humanoid robot control:

• Fully known parameters: All mass, inertia, and dynamic properties are precisely defined and unchanging
• Rigid links: Mechanical structure behaves predictably with well-defined kinematic chains
• Known center of mass (CoM): Position and dynamics of CoM can be calculated exactly
• Known joint torques: All actuator outputs are precisely controllable and measurable
• Predictable dynamics: System behavior follows rigid-body dynamics with high fidelity

These known parameters enable direct simulation-based trajectory planning, where trajectories designed in simulation can be executed reliably on hardware.

Powered Exoskeleton Control: Human-in-the-Loop Complexity

Unique challenges introduced by human involvement:

Priority Hierarchy: Safety > Comfort > Stability > Psychological factors > Efficiency

• Safety is paramount - any control strategy that risks injury to the pilot is unacceptable
• Comfort greatly affects pilot performance and willingness to use the system
• Stability must account for unpredictable human input and reactions
• Psychological factors include pilot confidence, cognitive load, and adaptation time
• Efficiency becomes secondary to human-centric considerations

Fundamental complications:

• Unpredictable dynamics: Human muscle activation, reflexes, and voluntary movements introduce unmodeled forces and moments
• Psychological factors: Pilot comfort, confidence, and fear responses significantly affect performance
• Model uncertainty: Combined human-exoskeleton dynamics cannot be precisely predicted due to variable human behavior

The Core Challenge: Unlike humanoid robots where control focuses on mechanical optimization, exoskeleton control must primarily optimize for human factors while maintaining mechanical performance.

This fundamental difference means that sim-to-real transfer for exoskeletons requires not just technical accuracy, but also deep consideration of human biomechanics, psychology, and comfort factors that are impossible to fully model in simulation.

---

1. The Reality Gap: From Simulation to Hardware

The transition from simulation to real hardware revealed fundamental gaps in our understanding of the physical system. To systematically identify and address these discrepancies, we conducted controlled experiments that isolated specific aspects of the walking behavior.

Single Leg Trotting: Revealing Timing Discrepancies

To better understand the sim-to-real gap, we analyzed single leg trotting motions that clearly revealed fundamental timing differences between simulation and hardware:

Key observations from the side-by-side comparison:

• Late toe-off: The real exoskeleton exhibits delayed toe-off compared to the simulation, indicating higher resistance during push-off phase
• Early heel contact: Ground contact occurs earlier in the real system than predicted by simulation, suggesting differences in leg compliance and descent dynamics
• Stance phase duration: The overall stance phase is extended in reality, affecting the rhythm and efficiency of the gait

Quantitative GRF Analysis: Measuring Timing Discrepancies

To quantify the timing discrepancies observed in the trotting analysis, we conducted detailed Ground Reaction Force (GRF) measurements throughout complete gait cycles:

Data synchronization methodology:

• GRF data was collected at 1000Hz using force sensors embedded in each foot
• Joint kinematics were synchronized with force measurements using hardware timestamps
• Gait phase boundaries were identified using threshold-based detection on vertical GRF signals
• Timing discrepancies were calculated as the difference between predicted and measured event occurrences

The timing discrepancy for each gait event was quantified as:

$\Delta t_{event}=t_{hardware}-t_{simulation}$

where heel-strike and toe-off events were detected using vertical GRF thresholds:

$t_{heel-strike}=\arg {\min }_t\{F_{GRF}(t)>F_{threshold}\}$ $t_{toe-off}=\arg {\min }_t\{F_{GRF}(t)<F_{threshold}\}$

The stance phase duration error was then calculated as:

$\Delta t_{stance}=(t_{toe-off}-t_{heel-strike}{)}_{hardware}-(t_{toe-off}-t_{heel-strike}{)}_{simulation}$

This quantitative analysis confirmed the visual observations from the trotting experiments and provided precise measurements for model refinement.

For more details about how the extensive data from the exoskeleton was collected, analyzed, and used to make real-time adjustments, please read my post: Trajectory Analysis & Optimization UI for Bipedal Robots (Real-Time Kinematics & Visualization)

Early Ground Contact and Gait Timing Mismatch

The single leg trotting analysis revealed a critical timing discrepancy that became one of our major challenges. The early heel-strike observed in the trotting gifs represented a fundamental problem that affected all walking behaviors.

Problem Analysis:

• In the real world, heel-strike occurred earlier than in simulation, resulting in shock to the user and deformation of the exoskeleton structure
• This mismatch was visually apparent in the trotting comparisons and quantitatively confirmed by comparing Ground Reaction Force (GRF) vs. time data from force sensors in the foot
• Root cause: The combined weight of the human pilot and exoskeleton, as well as unmodeled compliance in the foot and ground interface, led to earlier-than-expected ground contact

Diagnostic Approach:

• GRF data was synchronized with joint kinematics to precisely identify the timing of push-off and heel-strike events
• The difference in timing between simulation and hardware was measured and analyzed
• The trotting experiments provided a controlled environment to isolate these timing effects

Solution Development:

• The swing foot trajectory was modified in the MPC to increase ground clearance and adjust the timing of descent
• The physical model was further refined to include foot compliance and pilot mass distribution

These timing discrepancies highlighted the need for more accurate modeling of contact dynamics, leg compliance, and the combined inertial properties of the human-exoskeleton system.

---

2. Excessive Lateral Shifting and Base of Support

Initial Transfer: Primary Challenge

The first attempt to deploy an MPC-generated walking trajectory from simulation to the real WalkOn F1 exoskeleton clearly demonstrated the sim-to-real gap. While the trajectory was dynamically stable and met all constraints in simulation, the real robot's motion was uncomfortable, with excessive lateral shifting and unnatural transitions.

Technical context:

• The MPC was formulated as a constrained optimization problem, minimizing a cost function that included terms for CoM tracking, foot placement, joint velocities, and input torques, subject to dynamic and kinematic constraints.
• The simulation model assumed idealized actuation, perfect ground contact, and simplified mass distribution.

Observed issues:

• Excessive lateral shifting of the CoM before each forward step, required to move the projected CoM over the base of support.
• Uncomfortable and non-human-like motion for the pilot.

The Fundamental Challenge: Flat-Foot Walking in MPC

The issues observed in our initial implementation are endemic to most MPC-controlled humanoid robots, which rely on flat-foot walking strategies due to the computational complexity of multi-contact dynamics. Understanding this limitation was essential for developing our technical approach.

Why Most MPC Robots Use Flat-Foot Walking:

The MPC optimization problem for bipedal locomotion can be formulated as:

${\min }_{x,u}{\sum }_{k=0}^{N-1}\left[\Vert x_k-x_{ref,k}{\Vert }_Q^2+\Vert u_k{\Vert }_R^2\right]+\Vert x_N-x_{ref,N}{\Vert }_{Q_f}^2$

Subject to:

• System dynamics: $x_{k+1}=f(x_k,u_k)$
• Contact constraints: ${\varphi }_i(q_k)\ge 0$ for all contact points $i$
• Friction cone constraints: $\Vert F_{t,i}\Vert \le \mu F_{n,i}$

For flat-foot contact, the constraint set involves a single contact region per foot, making the problem computationally tractable. However, modeling the transition from heel-to-toe contact requires handling multiple simultaneous contact points with complementarity constraints:

$0\le {\varphi }_i(q)\perp {\lambda }_i\ge 0$

where ${\varphi }_i$ represents the gap function and ${\lambda }_i$ the contact force for each potential contact point. This dramatically increases the problem complexity from convex quadratic programming to mixed-integer programming, often making real-time solution infeasible.

For more details on MPC implementations for exoskeletons, please read my post: Model Predictive Control for Powered Lower-Body Exoskeleton Robots

Consequences of Flat-Foot Walking:

1. Excessive lateral CoM shifting: Before initiating swing, the flat foot must be entirely within the base of support, requiring large lateral CoM displacement: $\Delta x_{CoM}\approx L_{foot}/2+\text{stability margin}$

2. Large shocks at toe-off: Requiring high torque at the knee and ankle to initiate swing, causing mechanical stress and energy inefficiency

3. Flat-foot heel-strike: Causing large impact forces and discomfort for the user, as the entire foot contacts the ground simultaneously

4. Slow walking speeds: The prolonged double-support phase for CoM shifting limits step frequency: $f_{step}\le \frac{1}{t_{double-support}+t_{single-support}}$

5. High energy consumption: Large lateral accelerations increase metabolic cost: $E\propto m\cdot a_{lateral}^2\cdot d_{lateral}$

6. Non-human-like walking pattern: Abrupt transitions between stance and swing phases, lacking the smooth heel-to-toe progression of natural human gait

7. Pilot discomfort: Unnatural motion patterns cause vestibular conflicts and increased cognitive load

8. Poor adaptability: Rigid flat-foot strategies cannot adapt to terrain variations or dynamic disturbances

9. Limited maneuverability: Large turning radii due to CoM positioning constraints

The Cybathlon Speed Imperative:

The Cybathlon competition format imposed strict time constraints: teams had to complete 10 different locomotion tasks within a 10-minute time limit. This constraint made walking speed a critical competitive factor, as faster locomotion enabled teams to:

• Complete more tasks within the time limit
• Allocate additional time to complex or challenging obstacles
• Maintain competitive advantage over teams using slower flat-foot walking strategies

Teams relying on conventional flat-foot MPC approaches faced significant time penalties due to their inherently slow walking speeds, making speed optimization essential for competitive success rather than merely desirable.

The Challenge of Natural Contact Progression

In human walking, the point of contact transitions smoothly from heel to toe during stance phase, enabling natural weight transfer and propulsion. A single-point contact model cannot capture this biomechanical progression, while increasing the number of simultaneous contact points in the MPC makes the optimization problem exponentially harder and often unsolvable in real time due to the combinatorial explosion of contact states.

Solution Development

To address these fundamental limitations while maintaining computational tractability, we developed a novel dynamic contact strategy that enables natural heel-to-toe progression. The detailed implementation and technical approach for this solution are presented below in Section 7: "Implementing Dynamic 3-Point Contact: The Path to Human-Like Walking."

---

3. Leg Bending and Collision Risks

Problem Analysis

A critical safety issue emerged: during the swing phase, the swing leg would bend inwards, sometimes causing it to collide with the stance leg. This was not observed in simulation.

• Root cause: The real exoskeleton's ankle modules were heavy, introducing compliance and a moment arm that caused the swing leg to bend medially under load during the aerial phase.
• Simulation: The model did not account for this compliance or the mechanical deformation effects of the heavy ankle modules.

The following image demonstrates the mechanical deformation issue that posed serious collision risks during walking:

Key observations:

• Inward bending of swing leg: The swing leg exhibits significant medial deviation (δ(q)) during the aerial phase, deviating from the intended straight-line trajectory
• External torque effects: The heavy ankle modules create an external moment (τ_ext) that causes the leg to bend inward under gravitational and inertial loads
• Hip joint compliance: Further analysis revealed that significant bending was occurring at the hip joint, contributing to the overall medial deviation beyond the ankle effects
• Collision risk zone: The inward bending brings the swing foot dangerously close to the stance leg, creating potential for mechanical interference
• Unmodeled compliance: The deformation reveals compliance in the leg structure that was not captured in the rigid-body simulation model

The medial deviation during swing phase can be modeled as:

$\delta (q)=\frac{{\tau }_{ext}}{K_{joint}}+\frac{m_{ankle}\cdot g\cdot L_{moment}}{K_{structure}}$

where:

• $K_{joint}$ represents joint stiffness (hip and knee compliance)
• $K_{structure}$ represents structural stiffness of the leg segments
• ${\tau }_{ext}=m_{ankle}\cdot g\cdot L_{moment}+{\tau }_{inertial}$ is the external moment from ankle mass
• $L_{moment}$ is the moment arm from joint center to ankle module center of mass

Solution Approach

To address the leg bending and collision issues, we implemented a multi-faceted solution combining both software and hardware improvements:

Simulation Updates: Increased the distance between legs during swing phase in the simulation to provide additional clearance and prevent collisions

Hardware Solutions - Structural Reinforcement: Additional investigation revealed that substantial bending was occurring at the hip joint, contributing significantly to the overall leg deformation problem which was later fixed.

Validation and Testing:

• Logged collision events and correlated them with joint angle trajectories to validate improvements
• Systematically tested the combined software-hardware solution under various walking conditions

Results: Improved Leg Clearance and Safety

The implementation of the comprehensive solution approach resulted in significantly improved leg clearance and elimination of collision risks:

Key improvements achieved:

• Eliminated leg bending: The swing leg no longer exhibits major inward deviation, maintaining proper trajectory during aerial phase
• Proper leg separation: Safe clearance between limbs is consistently maintained throughout the gait cycle
• Stable stance leg: The stance leg remains properly aligned and stable during swing phase of opposite leg
• Enhanced safety: The pilot can walk confidently without fear of mechanical interference
• Improved sim-to-real correspondence: Real-world leg trajectories now closely match simulation predictions

This comprehensive solution represented an important milestone in achieving safe, reliable exoskeleton operation, enabling subsequent gait optimizations.

---

4. Transition to Continuous Walking

With the safety issues resolved, we transitioned from single-step commands to implementing continuous walking in our finite state machine. This required gait cycle management and seamless transitions between stance and swing phases:

Key developments in continuous walking:

• Finite State Machine Integration: Implementation of a robust state machine that could seamlessly transition between gait phases
• Gait Cycle Coordination: Automated coordination of left and right leg movements with proper timing and phase relationships
• Progressive Implementation: Initially implemented continuous walking with slow speeds and smaller step lengths to ensure pilot safety and comfort
• Gradual Performance Scaling: As the pilot became more accustomed to the continuous walking system, walking speed and step lengths were gradually increased to approach competition-ready performance levels

---

5. Jerky Motions: The Cost of Over-Constraint

Progress Achieved

At this stage of development, the exoskeleton had made significant progress in several key areas:

Improvements observed:

• Sustained gait cycles: The exoskeleton maintained consistent, repeatable walking patterns without manual intervention
• Better trajectory execution: Improved adherence to the planned foot trajectories compared to earlier attempts
• Stable stance phase: More reliable weight transfer and support during the stance phases
• Safety compliance: Collision risks had been eliminated and leg clearance was consistently maintained
• Continuous operation: The system could transition seamlessly between gait phases for sustained locomotion

Remaining Challenge: Motion Quality Issues

Despite these achievements, real-world trajectories still exhibited jerky, non-smooth motions that needed to be addressed for competition readiness. Hardware-only testing revealed the nature of these remaining issues:

Remaining challenges identified:

• Swing phase deceleration: The swing foot exhibits unnatural mid-air deceleration, appearing to "pause" during the forward step motion
• Cost function limitations: The MPC cost function weighting was not optimally tuned for smooth swing trajectories, prioritizing constraint satisfaction over natural motion
• Swing motion artifacts: The defined swing motion in the MPC created discontinuities that resulted in jerky transitions during the aerial phase

Problem Analysis

The jerky motions were traced to two main factors:

• Inefficient weighting of the cost function terms, leading to abrupt transitions between phases
• Over-constraining the MPC problem with too many (sometimes conflicting) constraints, making the solution sensitive and non-robust

Solution

The cost function was reformulated to prioritize smoothness by incorporating jerk minimization terms:

$J={\sum }_{k=0}^{N-1}\left[\Vert x_k-x_{ref,k}{\Vert }_Q^2+\Vert u_k{\Vert }_R^2+\Vert {\ddot{q}}_k{\Vert }_{W_{acc}}^2+\Vert {\stackrel{...}{q}}_k{\Vert }_{W_{jerk}}^2\right]$

where:

• $\Vert {\ddot{q}}_k{\Vert }_{W_{acc}}^2$ penalizes joint accelerations to reduce abrupt motion changes
• $\Vert {\stackrel{...}{q}}_k{\Vert }_{W_{jerk}}^2$ penalizes joint jerk to ensure smooth acceleration profiles
• $W_{jerk}\gg W_{acc}$ to heavily penalize jerky motions

Constraint optimization approach:

• Redundant or conflicting constraints were identified and removed
• The constraint set was reduced from the original formulation
• This reduction improved optimization feasibility while maintaining safety and task completion requirements.

---

6. Achieving Fast, Continuous, and Human-Like Walking

Performance Requirements

To be competitive at Cybathlon, achieving safe and comfortable walking was insufficient, the exoskeleton also needed to demonstrate fast, continuous locomotion with minimal pauses between steps.

Optimized Smooth Walking: Cost Function Refinement

After identifying the swing phase deceleration issues, we focused on optimizing the MPC cost functions and redefining the swing motion parameters. The result was a significantly smoother and more natural walking pattern:

Key optimizations implemented:

• Cost function rebalancing: Adjusted the weighting between different objective terms to prioritize smooth swing trajectories while maintaining stability constraints
• Swing phase motion redesign: Redefined the swing foot trajectory with continuous acceleration profiles, eliminating the mid-air deceleration artifacts
• Velocity continuity constraints: Added constraints to ensure smooth velocity transitions at the boundaries between stance and swing phases
• Simulation-reality alignment: Achieved much closer correspondence between simulated and real walking patterns, as evidenced by the side-by-side comparison

Results achieved:

• Elimination of the unnatural swing foot "pause" observed in previous iterations
• Smoother, more human-like leg movements during both stance and swing phases
• Better energy efficiency due to reduced jerky motions and unnecessary accelerations
• Improved overall walking rhythm and fluidity

This optimization marked a significant step toward achieving competition-ready, natural exoskeleton locomotion.

---

7. Implementing Dynamic 3-Point Contact: The Path to Human-Like Walking

With the jerky motion issues resolved and smooth walking achieved, analysis revealed remaining limitations in the flat-foot contact model. The system exhibited excessively large pushoff forces and required substantial knee torques to generate adequate toe-off propulsion. These large forces created uncomfortable transitions for the pilot and energy inefficiency. The next phase focused on implementing an advanced contact model to enable smoother, more human-like transitions through the complete gait cycle: stance → pushoff → swing → heel strike → stance.

Theoretical Foundation: Contact Point Modulation Strategy

The core idea of 3-point contact system lies in the systematic modulation of contact points during each gait cycle. The following diagram illustrates how the contact point transitions through five distinct phases during a single swing step:

Phase-by-Phase Contact Progression:

• Phase 0 (Initial Stance): Contact at center of foot, providing stable support during double-support phase
• Phase 1 (Push-off Preparation): Contact transitions toward toe region, preparing for propulsive push-off
• Phase 2 (Toe-off): Contact at toe point, maximizing forward propulsion and natural toe-off mechanics
• Phase 3 (Heel Strike Preparation): Contact point shifts to heel region in preparation for landing
• Phase 4 (Heel Strike): Contact at heel, enabling natural heel-first ground contact

This systematic contact modulation enables the biomechanically accurate heel-to-toe progression that characterizes natural human walking, while maintaining the computational efficiency required for real-time MPC optimization.

Implementation Strategy

Having established the theoretical foundation, the transition from optimized flat-foot walking to dynamic 3-point contact required careful integration of several components:

MPC Integration:

• Modified cost function to optimize for natural contact progression
• Constraint formulation to enforce single-contact-per-foot limitation
• Trajectory planning that accounts for varying contact locations during stance

Safety Considerations:

• Gradual transition from flat-foot baseline to ensure stability
• Extensive validation in simulation before hardware deployment

Refined 3-Point Contact with Pilot Integration

Combining all the previous optimizations, we achieved an effective walking strategy: refined 3-point contact working seamlessly with a human pilot. This represented the culmination of all our sim-to-real learnings and the best performing configuration:

Contact mechanics implemented:

• Dynamic contact point transitions: The contact point smoothly transitions from center → toe during push-off, heel during landing, and back to center during stance
• Phased contact progression: Heel strike → center stance → toe-off sequence mimics natural human gait biomechanics
• Real-time contact visualization: The simulation clearly shows the moving contact point (visible in the right panel) as the exoskeleton progresses through each gait phase
• Pilot-exoskeleton synchronization: The human pilot can naturally adapt to the exoskeleton's movement patterns, reducing cognitive load and improving comfort

Comprehensive improvements achieved:

• Biomechanically accurate locomotion: Natural heel-to-toe contact progression eliminated all flat-foot impact issues
• Smooth swing trajectories: Integration of optimized cost functions with 3-point contact eliminated swing phase artifacts
• Pilot adaptability: Enhanced swing trajectory smoothness made the system easily adaptable by the human pilot
• Competition readiness: This configuration provided the performance, comfort, and reliability needed for Cybathlon 2024

Key Achievement

The successful implementation of dynamic 3-point contact represented a fundamental advancement in exoskeleton locomotion, enabling:

• Natural heel-to-toe progression that mimics human biomechanics
• Substantial reduction in lateral shifting requirements
• Improved walking speeds approaching natural human velocities
• Enhanced comfort and naturalness for the pilot

This implementation of dynamic 3-point contact represented a significant technical advancement, demonstrating that successful sim-to-real transfer requires not just mechanical accuracy, but also biomechanically-inspired design approaches. However, achieving truly optimal performance required additional fine-tuning, adaptation to individual pilot preferences through extensive human-in-the-loop optimization, and further gait trajectory refinement, developments that culminated in the final competition-ready solution presented in the next section.

---

8. Final Results: Optimized Natural Walking

After 8 months of intensive optimization, iterative refinement, and extensive sim-to-real debugging, we achieved natural walking performance that closely approximates human locomotion. This final demonstration represents the integration of all our research and engineering developments:

Key achievements demonstrated:

• Systematic development: Eight months of optimization, testing, and iterative improvement led to this locomotion performance
• Effective human-robot integration: The pilot moves naturally and confidently, with minimal visible adaptation required
• Natural movement quality: The locomotion pattern closely resembles natural human walking enhanced by robotic assistance

The successful integration of multiple technical improvements resulted in locomotion that appears natural and effortless to the pilot. The effectiveness of the approach is demonstrated by the seamless walking performance achieved.

---

Conclusion: Key Learnings

Through this intensive 8-month journey from initial sim-to-real challenges to achieving perfect natural walking, several critical insights emerged that extend beyond this specific project to the broader field of robotics and human-robot interaction:

• Simulation ≠ Reality: Real-world dynamics, compliance, and human-robot interaction introduce complexities not captured in simulation. Accurate physical modeling and system identification are essential.
• Iterative, Data-Driven Optimization: Success required a continuous loop of modeling, real-world testing, data analysis, and refinement.
• Constraint Management: Over-constraining the MPC can make solutions infeasible or uncomfortable; judicious constraint selection and cost function tuning are critical.
• Contact Modeling: Realistic, yet computationally efficient, contact models are essential for smooth and safe walking.
• Pilot Feedback: Direct, structured feedback from the human pilot was invaluable in tuning for comfort and safety, and should be integrated into the optimization process.

---

This work was conducted at KAIST's ExoLab under the guidance of Prof. Kong.

Related Posts

---