The Proposal for an Artificial Intelligence Act, published by the European Commission in April 2021, marks a major step in the governance of artificial intelligence (AI). This paper examines the significance of this Act for the electricity sector, specifically investigating to what extent the current European Union Bill addresses the societal and governance challenges posed by the use of AI that affects the tasks of system operators. For this we identify various options for the use of AI by system operators, as well as associated risks. AI has the potential to facilitate grid management, flexibility asset management and electricity market activities. Associated risks include lack of transparency, decline of human autonomy, cybersecurity, market dominance, and price manipulation on the electricity market. We determine to what extent the current bill pays attention to these identified risks and how the European Union intends to govern these risks. The proposed AI Act addresses well the issue of transparency and clarifying responsibilities, but pays too little attention to risks related to human autonomy, cybersecurity, market dominance and price manipulation. We make some governance suggestions to address those gaps.