Can I pay contributions retroactively to recover the past contribution-exempted period or past postponed period, in order to increase pension benefits?

Yes, you can recover those periods by paying the exempted or postponed contributions retroactively for up to the past 10 years. Please note that a specific index will be imposed on your past contribution amount. That is, if you were granted the contribution exemption in a certain fiscal year (April to March) and if you wish to recover the contribution three or more fiscal years later, your old contribution amount is indexed by certain rate according to the age of the due contribution.
[Note] Please make sure to apply for the retroactive payments well in advance. If you apply just before the deadline, your payments may not be technically accepted in time.