Just before Frank gestured to AJ to bring the nightstand containing the gun near him, AJ said:
AJ: I'm gonna get out, and I'm gonna get help. Gonna let everybody know
what's goin' on down here. That fuckin' thing is gonna pay for what it did. Don't worry, okay? There are gonna be fucking cops swarmin' this place real fuckin' soon. Trust me, man.
Frank realized that his crimes, which he has kept hidden for years, will be revealed once AJ gets out and tells the police what was going on down there. Franks's sick mind made him realize that it's better to just kill himself, rather than face the consequences of his horrible crimes.
I was expecting Frank to shoot AJ instead.
Frank, who is bedridden and wasting away, has probably realized that "this is the end." Rather than prolong his agony, risk exposure and punishment (killing AJ will have the police looking for AJ in that house), he chose to just kill himself.