The emergence of ChatGPT, a generative artificial intelligence tool, has sparked a revolution in the finance industry, enabling individuals to interact with technology in natural language. However, the use of ChatGPT in finance presents a profound array of ethical considerations that demand careful scrutiny to ensure its responsible and ethical use. After a concise exploration of ChatGPT's applications in finance, this policy article delves into the ethical challenges arising from the use of ChatGPT in finance, including outcomes contaminated with biases, incorporation of fake information in the financial decisions, concerns surrounding privacy and security, lack of transparency and accountability in the decision-making processes and financial services, human job displacement, and the intricate web of legal complexities. Our article asserts that financial institutions employing ChatGPT must proactively devise strategies to confront these burgeoning challenges, mitigating their adverse effects on both individuals and society as a whole. Additionally, we propose relevant policies to tackle these ethical quandaries head-on. In essence, this article illuminates the imperative need for a meticulous ethical framework, facilitating an informed and responsible use of ChatGPT in the realm of finance, safeguarding the welfare of individuals and society. While our work significantly contributes to the research and practice of finance, we also identify future research avenues.