Plasmonic nanoparticles (NPs), predominantly gold (AuNPs), are easily internalised into cells and commonly employed as nanosensors for reporter-based and reporter-free intracellular SERS applications. While AuNPs are generally considered non-toxic to cells, many biological and toxicity studies report that exposure to NPs induces cell stress through the generation of reactive oxygen species (ROS) and the upregulated transcription of pro-inflammatory genes, which can result in severe genotoxicity and apoptosis. Despite this, the extent to which normal cellular metabolism is affected by AuNP internalisation remains a relative unknown along with the contribution of the uptake itself to the SERS spectra obtained from within so called 'healthy' cells, as indicated by traditional viability tests. This work aims to interrogate the perturbation created by treatment with AuNPs under different conditions and the corresponding effect on the SERS spectra obtained. We characterise the changes induced by varying AuNP concentrations and medium serum compositions using biochemical assays and correlate them to the corresponding intracellular reporter-free SERS spectra. The different serum conditions lead to different extents of nanoparticle internalisation. We observe that changes in SERS spectra are correlated to an increasing amount of internalisation, confirmed qualitatively and quantitatively by confocal imaging and ICP-MS analysis, respectively. We analyse spectra and characterise changes that can be attributed to nanoparticle induced changes. Thus, our study highlights a need for understanding condition-dependent NP-cell interactions and standardisation of nanoparticle treatments in order to establish the validity of intracellular SERS experiments for use in all arising applications.