If all men and women are born into a state of sinfulness, then what happens to infants when they die?
The question of how God accommodates the souls of children (or for that matter, the souls of invalids, mentally incapacitated, etc.) is not easy to answer, though Scripture does give us a limited answer. First, we must understand how all men are saved. Scripture tells us that all men are saved by faith in Jesus Christ, and by no other means. Without faith in the Savior's atoning sacrifice, no one may enter God's presence.
But faith itself is not something that men arrive at by their own power and reason.
Paul says in Romans 3 that no man chooses God - nor even seeks for God - by his own nature and will. On the contrary, the natural (unsaved) man is spirtually opposed to God and is an enemy of God (Rom 5:10). Paul later says in Romans 8:7 that the mind of natural man is hostile toward God and will not subject itself to God's decrees nor is even able to do so. Since man will never naturally seek after God nor believe the Gospel, Paul tells us in Ephesians 2:8 that our faith itself is "a gift" from God. Without God's intervention to change our hearts, all men would remain forever unbelievers until death.
Taking all this together, we see that salvation is only possible when God grants an unbeliever a gift of faith, which leads the recipient to respond positively to the gospel and trust in Jesus. From the new believer's point of view, he remembers making a decision to believe the gospel and profess faith in Jesus, yet in reality his confession was itself proof that God had already been working in his heart to prompt this confession. Just as Jesus told Peter after Peter confessed that Jesus was Lord:
This brings us back to the subject of infants.
Saving faith does not come as the result of a human intellectual process, and therefore it is not dependent upon our mental capacity or any other human, fleshly ability. Otherwise, our salvation could be said to depend - at least in part - on our own effort (e.g., our effort of evaluating and understanding the gospel), but Paul says in Eph 2:8-9 that God grants salvation through the gift of faith so that no man may boast before God. Therefore, faith is a spiritual process initiated by God and entirely apart from the workings of men.
As Paul says in Roman 9:
Obviously, believing adults will normally arrive at some level of understanding regarding their belief in Christ, but this process occurs only after the truth of the gospel has been placed in their hearts through the regenerative power of the Holy Spirit.
Consider these words of Paul:
Paul teaches that the Gospel is not received by those who are "perishing," but the One who shines the Light of the knowledge of Christ in our hearts is God Himself. So, faith is something that arrives by God's power and appointment in order to bring salvation to the unsavable, including you and I.
This means that God may, if He chooses, grant the gift of faith to anyone at any age, since God doesn't depend on our intellect or understanding. God may bring faith into the heart of an infant (or even in the womb, as He did for John the Baptist), though parents won't detect the faith of a child until some time later, when the child gains the capacity to express themselves, leading to a public profession in Christ and baptism.
On the other hand, if that child dies as an infant, we might never know that God had saved the child. Nevertheless, the child would still be saved by faith even though the child never obtained the physical and intellectual ability to express faith publicly. In a glorified body, that child would still have an opportunity to praise and glorify God in eternity.
This is the source of our present mystery. We can't know if God saves all children in this way or merely some children - either choice is within God's rights as sovereign Lord (see Rom 9:14-18).
We may choose to believe that God grants the gift of faith to some infants but not to all, in the same way He saves some adults but not all. On the other hand, we could choose to believe that God will save all children who die in infancy (or before a certain age). In either case, we know that God saves children in the same way He saves all men: not for their inherent goodness nor lack of sinfulness (for there is no person of any age who lacks sin), but rather all must be saved through the gift of faith.
Since scripture never tells us whether God saves some children or all children, we must trust this detail is unnecessary for a successful walk as Christians. Instead, we can know that God will do what is good and holy, and we look forward to a Day when we will understand it all perfectly (1Cor 13:12).