Holden might not be reacting to Kemper's immediate threat, but to his broader situation
You interpretation is that Holden is reacting to the specific threat from Kemper which has receded.
But this is a narrow frame to use to interpret his reaction. The broader context is that Holden has been testing out a somewhat academic theory about whether he can derive new insights from serial killers that can be used to help solve cases. He is thinking about gathering data and doing analysis, and those are not risky activities.
My interpretation of the panic attack in the context of the whole show and Holden's character arc is different. The key is that Holden has now been shaken out of his academic approach and the potential consequences of his work are now personal. To gather the data he needs he has to expose himself to people who are dangerous not just to gather evidence.
The panic attack occurs because he realises that he wasn't just gathering data but was exposing himself to serious potential threats. And was going to have to do it again if the work were to continue.
In addition, his personal interaction with Kemper may have crystallised directly just how dangerous Kemper was, which might have been a purely theoretical thing before they interacted directly.
Either way, it is the sudden realisation that the threats from serial killers are real, concrete, personal and persistent that triggered the attack, not the specific threat at that moment which had gone away once the door was closed.