You can prevent the Journalling Agent from auto-associating by setting the option to No in Tools & Settings > Application Settings
Auto-association is where the email received by the Journalling Agent is cross referenced between the Contact who sent the email and any open Leads, Jobs, Projects or Tickets.
On matching a Contact on any of these open records, Tracker places a link between the email activity and the Contact but also the latest open record for which they are the main contact point.