Fix #91369: Remove physics constraint references to avoid crash on undo

Fixes #91369, where a pointer to a deleted btTypedConstraint instance
was being dereferenced, causing a crash.

What was happening was:
1. When the animation starts the first time, BKE_rigidbody_rebuild_sim
  eventually calls btDiscreteDynamicsWorld::addConstraint, which in turn
  will store a pointer to the btTypedConstraint in
  btRigidBody::m_constraintRefs for each body in the constraint
2. When undoing, the btDynamicsWorld is deleted, then the Object
  containing the btTypedConstraint (taking the btTypedConstraint with
  it) - however, the pointer to the btTypedConstraint is still in the
  btRigidBody!
3. When playing the animation a second time, rigidbody_update_simulation
  will rebuild the simulation, which causes RB_body_delete to be called,
  which iterates over all the body's m_constraintRefs and dereferences
  the deleted pointer.

Co-authored-by: Eoin Mcloughlin <hkeoin@eoinrul.es>
Pull Request: https://projects.blender.org/blender/blender/pulls/126079
This commit is contained in:
Eoin Mcloughlin 2024-08-08 14:05:14 +02:00 committed by Lukas Tönne
parent 149825bc2a
commit 5ee7d02840

View file

@ -1120,6 +1120,13 @@ rbConstraint *RB_constraint_new_motor(float pivot[3],
void RB_constraint_delete(rbConstraint *con)
{
btTypedConstraint *constraint = reinterpret_cast<btTypedConstraint *>(con);
/* If the constraint has disabled collisions between the bodies, those bodies
* will have a pointer back to the constraint. We need to remove the constraint
* from each body to avoid dereferencing the deleted constraint later (#91369) */
constraint->getRigidBodyA().removeConstraintRef(constraint);
constraint->getRigidBodyB().removeConstraintRef(constraint);
delete constraint;
}