There are good arguments that he did and didn't know, but I believe he did know for the following reasons:
- His "Hail Mary" speech to Anthony partly conveyed his feelings of fear
- His understanding when Anthony was removed from the boat, with a simple, tragic "ah"
- Mario Puzo, in his novel Godfather Returns (released after the film) suggests he knew.
Explanation of scene:
Note: This answer refers to The Godfather II script for much of its content:
In the film, things are definitely left up to interpretation. He initially discusses the trip with his nephew Anthony, and says the following:
Anthony, ole buddy, your Uncle Fredo's gonna teach you how to catch the big fish. You know, when I was a kid, I did this amazing thing. I went out on a fishing trip; me and my brothers and my Pop, and no one could catch a fish except me. And this was my secret:
(confidentially) Every time I would put the line down I would say a "Hail Mary" and every time I said a "Hail Mary" I would catch a fish. Now, when it's sunset, we're gonna go out on the lake, and we're gonna try it.
When he actually tries to leave with Anthony, the script leaves his response deliberately vague:
Fredo: He's here; we're goin' fishing.
Connie: He can't go; Michael wants to take him into Reno.
Fredo: Ah. Okay, kid, you got to go to Reno with your Pop.
There's no indication here of panic, or fear - but that obviously doesn't mean they weren't there. When the moment itself comes, the script says the following:
Fredo and Neri are fishing, each with lines out. The VIEW MOVES
CLOSER, and we can hear Fredo as he holds the pole.
Fredo: ... the Lord is with thee. Blessed art thou amongst women, and blessed is the fruit of thy womb, Jesus.
The boat on the shimmery lake.
Fredo: ... Holy Mary, Mother of God, pray for us...
We hear a quiet, echoing GUNSHOT; and then silence.
We know Fredo has been terrified of his brother. Consider this scene from earlier in the film:
Michael: I know it was you, Fredo. You've broken my
Slowly, understanding, Fredo backs away from his brother, taking the
kiss another way.
Then later, when the presidential palace is under attack:
Michael moves quickly toward his car. He sees Fredo, watching him in
Michael: Come with me. It's your only way of getting out!
VIEW ON FREDO
Terrified of his brother, and what he knows; Fredo backs away into the
growing noise and confusion of the crowd.
This leads Michael to say the following to him later:
Michael: You're nothing to me now, Fredo; not a brother, not a friend, I don't want to know you, or what happens to you. I don't want to see you at the hotels, or near my home. When you visit our Mother, I want to know a day in advance, so I won't be there. Do you understand?
Therefore, it's clear Michael despises him at this moment in time. However, after the death of their mother, he allows Fredo to embrace him (albeit rather stiffly).
I believe there are a few possible interpretations here.
Firstly, Fredo says the Hail Mary to bring him luck - and in his words to Anthony perhaps suggests he has been saying the Hail Mary to himself ever since he betrayed his brother, knowing his time would soon be up. When Connie says Anthony can't respond, he gives a simple "Ah". There is no argument, just quiet, scared understanding. He does say the Hail Mary on the lake, perhaps knowing the end has come.
Secondly, another piece of evidence that support this is Fredo's knowledge of how vicious his brother is. Other people who have betrayed him have met death, and it's arguable he knew he had limited time left given this.
However, none of this is conclusive.
There are some good counter arguments. Perhaps when Anthony was removed, he didn't argue because he was scared of his brother and knew if Michael wanted him, he had to let him go - there's a big difference between being scared to argue with Michael and thinking he would be murdered.
Similarly, following their mother's death (and at Connie's request), Michael appeared to have relented and offered some reconciliation. He embraced him. He also allowed him to see his children. Fredo is shown in the film to be quite gullible, and it's certainly plausible he doesn't understand this deception even at the very end, believing he was redeeming himself in the eye's of his brother.
I genuinely think both interpretations hold their own and can be argued for.
However, one thing sways the discussion for me - the novel, Godfather Returns. In it (released after the film was released). in it, the scene where Anthony is removed from the fishing boat is described as follows:
Right as they were about to shove off, Aunt Connie came running down
the dock, shouting that Anthony's father needed to take him to Reno.
Anthony started to complain, but Uncle Fredo got a hard look on his
face and said that Anthony had to go. He promised to take him tomorrow
instead. The boy, devastated, nodded and tried not to let on.
It certainly seems in the novel that Mario Puzo wanted to convey that Fredo knew the end was nigh. Given this interpretation, and how closely Coppola tried to reflect much of the novel, I believe Coppola's intention in the film was to portray Fredo as realising, at the moment Anthony was removed from the boat, with a simple "Ah", that his end had come.