Why did increasing budget of Facebook ad set lead to increase in Cost Per Purchase?

Digging into the data I saw there was a clear decrease in the CTR trend from the day in which I increased the ad set's budget and also in the conversion rate from click to app install.

Your finding above makes me believe that you may have increased the budget too aggressively, leading to the ad sets entering into learning phase and negatively impacting performance. You also may have reached the outer limits of your optimal audience, which would mean targeting needs to be adjusted. 

This is a good question. I'm assuming these ad sets are running with no bid control (eg. minimum cost)?

I believe the reason this happens is that Facebook expands its targeting for the ad set in order to spend the higher daily budget, and as a result, it exposes the ad to people for whom it is less relevant. This happens right away in an exploratory phase, but it can also result in a new baseline being established if the ad set isn't subsequently changed.

I'd recommend setting a cost control; more information on that here: Understanding bid strategies and cost control in Facebook advertising