Microsoft.Jet.OLEDB.4.0 provider is not registered on the local machine
I found a solution for this problem. The issue I described in my question occured basically due to the incompatibility of the Microsoft.Jet.OLEDB.4.0 driver in 64 bit OS.
So if we are using Microsoft.Jet.OLEDB.4.0 driver in a 64 bit server, we have to force our application to build in in 32 bit mode (This is the answer I found when I did an extensive search for this known issue) and that causes other part of my code to break.
Fortunately, now Microsoft has released a 64 bit compatible 2010 Office System Driver which can be used as replacement for the traditional Microsoft.Jet.OLEDB.4.0 driver. It works both in 32 bit as well as 64 bit servers. I have used it for Excel file manipulation and it worked fine for me in both the environments. But this driver is in BETA.
You can download this driver from Microsoft Access Database Engine 2010 Redistributable