The second angle from which we can address the claim that recreation is selfish and wasteful is to look at New Testament examples.
We would surely consider Jesus and his apostles as examples of working hard for the kingdom of God. And yet, after the apostles finished a hard preaching, teaching and healing tour, Jesus told them to come away and rest (Mark 6:30-31).
When the Apostle Paul was on his way to Jerusalem, knowing that he was facing prison and possibly death, he took time out to walk from the city of Troas to a city named Assos by himself (Acts 20:13-14). This hike is beautiful, and although walking slowed down his journey, he likely used this time to help prepare himself for what was coming.
We should note that in the incident mentioned above, when Jesus took his apostles aside to rest, a large crowd soon found them. Jesus ended up teaching them and then miraculously feeding them. We learn from this that sometimes we need to postpone or delay our times of refreshment when ministry needs arise. Having said that, however, we need to realize that ministry needs will always be out there. We cannot address them non-stop and continue doing so indefinitely. If we break down physically, emotionally and spiritually, we may end up sidelined for a long period.