Nope, the Great Pyramid is not just filled with rubble. Neither are the other pyramids alongside it on the Giza Plateau, nor the slightly older pyramids further south at Saqqara and Dashur.
It’s not at all unreasonable to suspect that, though, as it would have made building them much easier. But when you’re there in person you can actually see evidence of the structure, style, and substance of the interior blocks of the pyramids too in various ways.
First, explorers, rulers, and and other curious or greedy characters over the millennia have hacked into the sides of several pyramids, including the Great Pyramid. In fact, the current entrance to the Great Pyramid for visitors today isn’t the original entrance. It’s an opening that was bored into the monument in more “modern” times to access the interior passageways and chambers, and those tunnels had to go through through solid stone, not rubble, as you can see when you enter this “newer” entrance.
Second, when you’re in the passageways and interior chambers, you can look up and see the solid blocks above and around you when you’re smack dab in the middle of the monument. Of course you could say that maybe they made the walls surrounding the interior chambers out of solid stones along with the exterior walls, but I think the first point above showed how that theory can be proven wrong from all the boring, tunneling, and hacking of new openings and tunnels that occurred much later and which found solid stone the entire way through.
Lastly, while not necessarily the case with the Great Pyramid, you can see more solid interior stones in some of the older Pyramids that haven’t held up as well and have crumbled more than 1 or 2 layers deep. This can even be seen at Giza if you look at the Queens Pyramids beside the Pyramids of Khufu (the Great one) and Menkaure. Some of the Queens Pyramids are eroded almost completely away such that you full view of the interiors of those that were solid stone too.
So in sum, that’s a very good question, but we know from seeing where others have bored deep into these massive monuments and left new holes and tunnels through random sample portions of the monuments, as well as from similar pyramids crumbling and revealing interior layers of solid stone blocks, that the pyramids were indeed massive, ingeniously-engineered structures made of solid stone.