Question: What is the most challenging thing about implementing and using agile?
The amount of self discipline needed to be successful with the approach.
One big difference between plan driven and agile approaches is the amount of prescriptive guidance on which roles do what tasks at what time in the project. Many processes based on the plan driven paradigm (also referred to as "waterfall") assume the more that is spelled out for the team with respect to what is done when by whom, the more predictable and successful the project will be. The longer team members work under this structure, the more dependent they become on this guidance to effectively tell them what to do next, whether they intend it or not.
When teams switch from that environment to one in which the only guidance is a set of fairly simple rules designed to guide complex behavior, the initial reaction is that all the guidance they had depended on had disappeared. Team members grope around for a little looking for direction until they realize that they can rely on their knowledge and expertise to know what to do next, and that the team members themselves are responsible for keeping each other moving.
Most team members will find that they have the necessary self discipline to survive and thrive in an agile environment. Some will discover that they really need the very prescriptive structure that is offered by the more plan driven approaches. These team members will either get help from the rest of the team who will provide that structure, or they will self select off the team, which is ok. This style of working is not for everyone, but it can be very effective for most.
The above discussion points to a related challenge – teams working on agile projects need to have a certain combination of skills and experience to know what they should do/not do when that guidance is not there. The entire team does not need to be made up of super stars, but there should be a few members on the team that have a fairly good understanding about what needs to be done.