In vitro methodologies to evaluate nanocarriers for cancer treatment: where are we?

Abstract

Cancer treatment continues to face significant challenges, including lack of selectivity, systemic toxicity, and the adaptive resistance mechanisms of cancer cells, which limit the effectiveness of conventional therapies. Nanomedicine offers a promising solution, using nanocarriers for targeted drug delivery, improved bioavailability, and reduced off-target effects due to properties such as the enhanced permeability and retention (EPR) effect and advanced surface modifications. Additionally, the integration of theranostic capabilities allows for real-time monitoring of treatment efficacy. However, the clinical translation of nanocarriers remains restricted due to the limitations of existing predictive models. Traditional two-dimensional (2D) in vitro models often fail to replicate the complexity of the human tumor microenvironment (TME), leading to discrepancies between preclinical and clinical outcomes. More sophisticated models have been developed to address these challenges, including three-dimensional (3D) tumor spheroids, organoids, and microfluidic tumor-on-a-chip (ToC) systems. These models offer a more accurate representation of the TME, enabling better assessment of nanoparticle penetration, retention, and therapeutic effects, while also reducing reliance on animal models. This review provides a comprehensive analysis of the in vitro models used to evaluate the anti-tumoral effects of nanocarriers alongside the methodologies employed to assess their safety and efficacy. Specifically, we explore the evolution from 2D monolayer cultures to advanced 3D systems, including tumor spheroids, organoids, and ToC platforms, and delve into the main methodologies employed to evaluate nanoparticle behavior, including cellular uptake mechanisms, cytotoxicity assays (e.g., MTT, WST-1/WST-8, LDH, Live/Dead assays), cell death mechanisms (e.g., apoptosis, necrosis, and autophagy), signaling pathways exploring gene expression analysis (qRT-PCR, RNA-seq, microarrays) and protein expression analysis (western blot, immunochemistry, mass spectrometry), oxidative stress evaluation, and cell migration/invasion assays (e.g., scratch/wound-healing, transwell, and microfluidic chip-based models). Furthermore, we cover clonogenic assays for assessing long-term cell survival, and cytoskeleton evaluation to understand how nanoparticles affect cell structure. By highlighting the advantages and limitations of these models and methodologies, this review aims to guide researchers in selecting the most appropriate experimental approaches, ultimately supporting the development of more effective nanomedicine-based cancer therapies.

Graphical Abstract

Introduction

Clinical challenges involved in cancer treatment, especially the lack of selectivity and consequent systemic toxicity, significantly limit the effectiveness of existing therapies, resulting in suboptimal therapeutic outcomes. Furthermore, the adaptive nature of cancer cells often leads to the development of resistance mechanisms, including the overexpression of efflux pumps and alterations in drug targets, which decrease the long-term efficacy of standard treatments (Emran et al. 2022).

Nanomedicine has emerged as an advancing field in cancer treatment due to its numerous advantages over conventional treatment. Nanocarriers can provide targeted delivery of therapeutic agents directly to tumor cells by taking advantage, for instance, of the enhanced permeability and retention (EPR) effect, which is a characteristic of the tumor microenvironment (TME). This effect occurs because tumors have leaky blood vessels and poor lymphatic drainage, allowing nanoparticles (NPs) to accumulate more easily in the tumor tissue compared to healthy tissues. As a result, NPs can deliver higher concentrations of drugs specifically to the tumor site, reducing damage to healthy cells and enhancing the overall effectiveness of the treatment (Kizhakkanoodan et al. 2023). However, the relevance of this effect in humans has been challenged by the scientific community. Although the EPR effect has been demonstrated in certain human tumors, its extent varies depending on both patient-specific factors and tumor heterogeneity (Harrington et al. 2001). Moreover, the effect tends to be more pronounced in small animal xenograft tumor models commonly used in preclinical studies than in tumors growing in the complex and less predictable human TME (Petersen et al. 2016).

Another advantage of nanocarriers is their ability to encapsulate a diverse range of therapeutic molecules, including hydrophobic drugs, nucleic acids (RNA, DNA), proteins, and peptides, enhancing their solubility, stability, and bioavailability. These carriers can be engineered to provide controlled release, ensuring a sustained therapeutic effect at the tumor site (Gharpure et al. 2015; Sánchez-Moreno et al. 2018). The multifunctional capabilities of NPs, such as surface modification with targeting ligands (e.g., antibodies, peptides), also enable therapies to be tailored to specific cancer phenotypes (Bandyopadhyay et al. 2023). Additionally, by integrating diagnostic and therapeutic functions into a single platform (termed theranostics), nanomedicine facilitates real-time monitoring of treatment efficacy, thereby optimizing therapeutic interventions (Gharpure et al. 2015; Sánchez-Moreno et al. 2018).

Despite the advancements in the design and application of nanocarriers for cancer therapy, their clinical translation remains restricted. One of the obstacles is the inadequacy of existing predictive models in evaluating both the efficacy and safety of these nanocarriers. The two-dimensional (2D) in vitro models lack the complexity of the TME, which can lead to an overestimation of therapeutic efficacy (Pickl and Ries 2009). In parallel, in vivo pre-clinical models (e.g., xenograft mouse models) also have significant limitations. These models do not fully recapitulate the human immune system, tumor heterogeneity, or the pharmacokinetics and biodistribution profiles seen in human patients, often resulting in poor predictive value for clinical outcomes. For instance, differences in NPs uptake, circulation time, and clearance between mice and humans can lead to promising preclinical results that do not translate into human trials. This discrepancy is exacerbated by the EPR effect, which is more pronounced in murine models than in humans (Bareham et al. 2021; Abdolahi et al. 2022).

These limitations have encouraged the development of more sophisticated and physiologically relevant in vitro models, including three-dimensional (3D) tumor spheroids and tumor organoids. They offer more realistic TME, allowing for a better assessment of NPs penetration, retention, and effects (Pickl and Ries 2009). Additionally, microfluidic systems, such as tumor-on-a-chip (ToC) technologies, are being explored to model the dynamic interactions between cancer cells, stromal components, and immune cells under controlled flow conditions that mimic in vivo physiology (Liu et al. 2021b). These models not only provide a better representation of the TME but also enable high-throughput screening of nanocarriers.

Furthermore, the increasing importance of 3D in vitro models in drug discovery reflects a paradigm shift toward more physiologically relevant and ethically aligned testing platforms. Legislative changes worldwide, such as the FDA Modernization Act 2.0, are driving this transition by eliminating mandatory animal testing requirements. Signed into law on December 29, 2022, as part of the Consolidated Appropriations Act of 2023, the FDA Modernization Act 2.0 (S.5002) amended the Federal Food, Drug, and Cosmetic Act (FD&C Act) to remove the mandatory requirement for animal testing in drug development. It allows for the use of alternative methods, such as 3D cell cultures and computational models. While it does not ban animal testing, it allows drug sponsors to use alternative non-animal models when demonstrating the safety and efficacy of new drugs, and aligns with global efforts to accelerate the adoption of innovative technologies and reduce animal testing (U.S. Congress. 2022).

This review provides a comprehensive analysis of the main in vitro models used to investigate the anti-tumoral effects of nanocarriers, encompassing traditional 2D monolayers and advanced 3D systems such as tumor spheroids, organoids, and ToC platforms. While several reviews have addressed the applications of nanocarriers in cancer therapy, focusing on their physicochemical properties, targeting strategies, and preclinical performance, an integrated perspective on the evolution of in vitro models and the methodologies used to evaluate NP-based therapies remains scarce. To bridge this gap, we explore not only the various in vitro models but also the key methodologies employed to assess NP interactions with tumor cells, including cellular uptake, cytotoxicity, and therapeutic efficacy. This review provides a more current perspective on the technologies used to evaluate NP interactions with in vitro tumor models, highlighting recent advances and methodological innovations. For clarity, the terms nanocarriers and NPs will be used interchangeably, encompassing a broad range of colloidal structures such as lipid-based vesicles, dispersed droplets, and polymeric or inorganic NPs. By critically evaluating the advantages and limitations of these models and methodologies, this review aims to provide researchers with a structured framework to support selecting the most appropriate experimental approaches, ultimately advancing the development of more effective nanomedicine-based cancer therapies.

In vitro models for cancer research

In vitro models for cancer research have become indispensable tools for studying tumor biology and evaluating the efficacy of anti-tumor therapies, including nanocarriers. The TME is highly complex, comprising not only cancer cells but also a diverse range of other cell types, such as cancer-associated fibroblasts, immune cells (including lymphocytes and tumor-associated macrophages), pericytes, and endothelial cells. Additionally, the TME is characterized by specific spatial organization and is surrounded by an extracellular matrix (ECM) that exhibits diverse mechanical and physicochemical properties (Balkwill et al. 2012). The ECM, which consists of a network of proteins, glycoproteins, glycosaminoglycans, proteoglycans, and growth factors, influences cell behavior within the tumor. Changes in ECM composition and stiffness can significantly impact tumor progression (Lu et al. 2012). Reproducing this complexity in vitro is a significant challenge, creating a need for the development of advanced 3D models that can more accurately replicate the dynamic interactions within the TME.

This section explores the evolution of in vitro cancer models, ranging from traditional 2D cell cultures to more advanced 3D systems. The models discussed here include 2D cell cultures (monolayers), tumor spheroids with single and multiple cell lines, tumor organoids, and ToC platforms. Each of these models offers advantages and limitations in replicating the complexity of the TME, with increasing sophistication as they evolve toward more representative systems for cancer research and nanocarrier testing.

Two-dimensional (2D) cell-culture (monolayers)

Monolayer cell culture models, which involve the growth of cells as a single layer on a flat surface, have been extensively used in cancer research and nanomedicine development due to their simplicity and ease of use. In this setup, cells are unable to stack on top of one another, limiting cell–cell interactions to their periphery. While monolayer cultures are widely used due to their low cost and easy manipulation, they capture only a limited aspect of cellular behavior and exhibit significant limitations, including altered cell interactions and the lack of 3D architecture (Garnique et al. 2024).

These models can employ both primary cultures, derived directly from patient or animal tissues, and immortalized cell lines, which are genetically modified or naturally adapted to proliferate indefinitely. Primary cultures offer the advantage of retaining the original phenotypic and genotypic characteristics of the tissue, providing a closer approximation of in vivo cellular behavior and tumor heterogeneity. However, they are limited by their finite lifespan, batch variability, and susceptibility to phenotypic drift over time (Langdon 2004). Figure 1A shows the typical 2D cell culture in a Petri dish. Moreover, 2D models fail to replicate essential aspects of tumor biology, such as the heterogenicity of TME (Fontana and Santos 2021), reducing their predictive value for clinical outcomes. Cells cultured in monolayers can also lose some intrinsic functions; for instance, HepG2 cells lose a considerable amount of CYP450 enzyme expression and mRNA activity, limiting their use for drug toxicity screening and liver function evaluation (Breslin and O’Driscoll 2016). However, when cultured in 3D, the same cells restore CYP3A4 mRNA expression and enzymatic activity similar to in vivo environment, evidencing that a more complex setting enables a condition closer to a living organism (Ramaiahgari et al. 2014).

Fig. 1

Schematic representation of monolayers (2D) and spheroid (3D) cell culture, where cell type 1 depicts a tumor cell and cell type 2 exemplifies either another tumor cell type or non-tumor cell. A Monolayer tumor cell culture—where cells are grown in a two-dimensional environment, providing simplicity and ease of use but lacking the 3D architecture and microenvironmental complexity of in vivo tissues; B Multicellular tumor spheroid—single-cell line spheroids are 3D cell aggregates that better mimic tumor architecture and gradients. From outer to inner layers, spheroids have a proliferative zone where cell cycle is progressing, a quiescent zone where cells are not dividing, and a necrotic zone, where cells are either dead or in the process of cell death; C Heterotypic multicellular tumor spheroid—Co-culture spheroids can incorporate multiple cell types, such as stromal or immune cells, to simulate intercellular interactions within the tumor microenvironment

Despite these limitations, 2D cultures remain a well-established methodology in cancer research due to their simplicity and practicality, answering some fundamental questions about cancer biology and the cytotoxicity profile of new treatments. For NPs, the most common application of monolayers is the evaluation of cytotoxicity, providing simple models for understanding the cytotoxicity of nanomaterials due to their low cost, high throughput, and high degree of environmental control (Bromma et al. 2023). Other applications include the assessment of NP internalization pathways, effects on cell migration, and interactions between NPs and specific cell proteins (such as collagen) (Meng et al. 2016; Figueiredo et al. 2017; Tao et al. 2017; Salata et al. 2021; Passos et al. 2023; Hirokawa et al. 2024).

However, with advancements in more reliable and complex in vitro models, monolayers should be regarded as preliminary steps in research. When comparing the cytotoxic response of NPs in monolayers versus 3D models, it is commonly observed that monolayer cultures exhibit higher sensitivity to treatments, while 3D models often show lower susceptibility. This phenomenon is primarily attributed to the protective nature of the outer cell layers in spheroids, which shield the inner layers from drug exposure, thereby reducing treatment efficacy (Chia et al. 2015; Chen et al. 2019). An example of this effect was demonstrated by Fukumori et al. (2023) in HCT-116 colorectal cancer spheroids treated with a multiple nanoemulsion containing tributyrin and tripropionin. The study found that the IC₅₀ values in spheroids were 9.5 and 12.4-fold higher, respectively, compared to the monolayer cultures (Fukumori et al. 2023). Similarly, Passos and collaborators (2024) reported a 3.3-fold increase in the IC₅₀ values of tributyrin-containing NLC with paclitaxel in MCF-7 breast cancer spheroids compared to monolayers (Passos et al. 2024).

Tumor spheroids—single cell line

Multicellular tumor spheroids are 3D models that more accurately simulate the interactions among cells and between cells and ECM (Fig. 1B). While these models are more expensive, laborious, and time-consuming compared to monolayers, they remain more cost-effective and less time-consuming than animal models. Moreover, they address ethical concerns while providing a closer representation of the TME. For example, 4T1 cells grown in 3D culture exhibit increased expression of Abgc2 (an ATP-binding cassette transporter associated with drug resistance) and reduced levels of Bax (a pro-apoptotic protein), indicating a more resistant phenotype compared to monolayer cultures (Priwitaningrum et al. 2023). Additionally, these cells also undergo epithelial-mesenchymal transition in spheroids, mirroring the behavior of in vivo tumors (Priwitaningrum et al. 2016). This 3D model has been applied to various cancer types, including colorectal cancer (Rane and Armani 2016; Tchoryk et al. 2019; Bauleth-Ramos et al. 2020; Rosso et al. 2024), glioblastoma (Niora et al. 2020), breast cancer (Estrada et al. 2016; Priwitaningrum et al. 2016, 2023; Salata and Lopes 2022; Dartora et al. 2022), cervical cancer (Takechi-Haraya et al. 2017), prostate cancer (Souza et al. 2018), pancreatic cancer (Priwitaningrum et al. 2016), Ewing tumor (Lawlor et al. 2002), non-small cell lung carcinoma (NSCLC) (Boghaert et al. 2017), melanoma (Albanese et al. 2013), and bone osteosarcoma (Carofiglio et al. 2024).

Spheroids can be produced using methods that prevent cell attachment to culture surfaces, forcing cells to interact with each other and creating a 3D, usually spherical structure. A summary of some spheroid formation methods, as well as their advantages and limitations, is displayed in Table 1. Structurally, spheroids are characterized by three distinct regions (Fig. 1B, C): a proliferative zone at the periphery, a quiescent zone in the middle, and a necrotic core at the center. This organization closely resembles the arrangement of in vivo tumors and is influenced by spheroid size. For instance, Ballangrud et al. (1999) reported that LNCaP (prostate cancer) spheroids with a diameter of 400 μm developed a non-proliferating core, while 600 μm spheroids exhibited a necrotic core. In contrast, no such regions were observed in smaller 200 μm spheroids (Ballangrud et al. 1999). Similar results were reported by Boghaert et al. (2017) for NCI-H1650 NSCLC spheroids, where proliferative, quiescent, and necrotic zones were evident only in larger spheroids (800–1200 μm), while smaller spheroids (200–400 μm) displayed a more homogeneous cell population. These size-related differences are associated with nutrient and oxygen gradients that are created due to their limited diffusion into the spheroid since these are non-vascularized structures (Boghaert et al. 2017).

Table 1 Overview of techniques for obtaining tumor spheroids, their principles, advantages and limitations

It is widely recognized that cellular structure influences gene expression. For example, Lawlor et al. (2002) demonstrated that Ewing tumor cells grown in spheroids more closely resemble primary tumors in morphology, cell–cell interactions and proliferative rates. Spheroid cells developed tight junctions similar to those in primary tumors, while monolayer cells exhibited poor cell–cell junctions and higher proliferation rates. Differences in gene expression, including cyclin D1 levels, were observed between the two models: in monolayers, high cyclin D1 expression was serum-dependent, whereas in spheroids, it was influenced by cell–cell adhesion. Additionally, ERK1 and ERK2 phosphorylation was consistently elevated in spheroids, regardless of serum presence, whereas monolayers showed increased phosphorylation only in the presence of serum (Lawlor et al. 2002). These findings suggest that spheroids provide a more accurate model for Ewing tumor and underscore the role of cell–cell adhesion in shaping the observed differences between monolayer and spheroid cultures.

Such differences in gene expression between 2 and 3D models often lead to important variations in treatment outcomes. For example, Pickl and Ries (2009) reported a stronger antiproliferative effect of trastuzumab in spheroids than in monolayers for SKBR-3 and SKOV-3 cells, representing breast and ovarian cancer cell lines, respectively. This effect was attributed to the increased formation of HER2 homodimers in spheroids, which are the primary target of transtuzumab, compared to the HER2/HER3 heterodimers observed in monolayers (Pickl and Ries 2009). Similar differential transcriptional profiles between spheroids and monolayers have been observed in NSCLC cells, where gene expression varied depending on cell confluence in monolayers and spheroid size (Boghaert et al. 2017). Additional evidence comes from 4T1 murine breast cancer cells, which showed reduced E-cadherin expression in 3D cultures compared to monolayers, further supporting the role of 3D architecture in driving a phenotype more reflective of in vivo tumors (Priwitaningrum et al. 2016).

It is also common for cytotoxic drugs to show reduced activity in spheroids compared to monolayers, likely due to the greater complexity and diffusion barriers in 3D models. Methods to increase drug penetration in spheroids often aid cytotoxicity. For example, Dartora et al. (2022) found that a nanoemulsion significantly increased rhodamine penetration in MCF-7 spheroids, which correlated with enhanced piplartine cytotoxicity (Dartora et al. 2022). Further differences in treatment response were reported by Tung et al. (2021), who observed that A431/H9 mesothelin-expressing human epidermal carcinoma monolayers were more susceptible to 5-fluorouracil than spheroids, likely due to a higher proportion of quiescent cells in spheroids. Conversely, spheroids were more sensitive than monolayers to tirapazamine, a hypoxia-activated cytotoxin, due to the hypoxia core typical of larger spheroids (Tung et al. 2011).

The limited NP penetration in tumors, which cannot be evaluated in monolayer models, is a broadly acknowledged challenge in nanotechnology research. Numerous studies have demonstrated that smaller NP sizes result in greater penetration in 3D spheroid models (Goodman et al. 2007; Grainger et al. 2010; Priwitaningrum et al. 2016, 2023; Tchoryk et al. 2019; Niora et al. 2020). For instance, Tchoryk et al. (2019) investigated the penetration of NPs into HCT116 colorectal cancer spheroids using polystyrene (PS) NPs of varying sizes (30, 50 and 100 nm) and surface chemistries alongside polymer-based NPs, including poly(glycerol adipate) (PGA) and PEGylated PGA derivatives. Smaller PS NPs (30 nm and 50 nm) exhibited superior penetration, including the spheroid core, while larger 100 nm NPs were mostly confined to the periphery. Surface chemistry significantly influenced penetration, with unmodified NPs outperforming aminated and carboxylated NPs, likely due to reduced electrostatic interactions with ECM components. PGA NPs (− 100 nm) achieved rapid and deep penetration, which was further enhanced by PEGylation, indicating that polymer flexibility and reduced ECM interactions are important for improved diffusion (Tchoryk et al. 2019).

These results correlate with the findings from Takechi-Haraya et al. (2017), who examined the influence of liposomal membrane rigidity on their penetration into HeLa spheroids. Their results indicated that liposome penetration depends on composition, as vesicles with higher bending moduli (i.e., more rigid) presented higher penetration, suggesting an influence of membrane rigidity on diffusion efficiency through the intercellular spaces and cellular uptake. Besides, NP PEGylation, which attributed negative charge to liposome surface, seemed to reduce penetration efficiency compared to unmodified liposomes of similar compositions and rigidity (Takechi-Haraya et al. 2017). Goodman et al. (2007) similarly investigated NP size and ECM integrity in SeHa (cervical cancer) spheroids, using carboxylate polystyrene NPs. They observed that smaller NPs (20–40 nm) achieved superior penetration, particularly when ECM integrity was disrupted by collagenase treatment. Collagenase-coated NPs penetrated deeper into the spheroid core, suggesting that the ECM limited NP diffusion through the 3D structure (Goodman et al. 2007).

Niora et al. (2020) further examined NP penetration in glioblastoma spheroids, comparing liposomes (Lip), PEGylated liposomes (PEG-Lip), lipoplexes (LPX), reconstituted high-density lipoproteins (rHDL) and polystyrene NPs (PNPs). rHDL, the smallest NPs examined, achieved the highest accumulation in the spheroid core. Notably, PEG-Lip presented greater penetration depths compared to unmodified liposomes, likely due to reduced interactions with the cell surface mediated by PEGylation. In contrast, LPX, despite being similar in size to unmodified liposomes and PEG-Lip, showed reduced penetration efficiency, which was linked to its positive surface charge (Niora et al. 2020). Grainger et al. (2010) also reported size-dependent NP penetration in spheroids, demonstrating that smaller NPs (20–40 nm) and negatively charged (carboxylate) NPs penetrated more effectively than larger or cationic counterparts. Additionally, pulsed ultrasound enhanced NP penetration in a time- and duty-cycle-dependent manner, offering a potential strategy to overcome diffusion barriers (Grainger et al. 2010). These findings, regarding the relationship between smaller NPs and higher penetration in spheroids, are consistent with in vivo data reported by Cabral et al. (2011), where smaller polymeric micelles more effectively penetrated pancreatic adenocarcinoma (BxPC3) tumors in BALB/c nude mice, reinforcing the suitability of tumor spheroids in evaluating NP penetration before transitioning to in vivo studies (Cabral et al. 2011).

In addition to polymeric and lipid-based systems, inorganic NPs have also been explored for tumor penetration using spheroid models. Carofiglio et al. (2024) utilized a 3D spheroid model composed of MG-63 osteosarcoma cells to evaluate NP penetration. The NPs employed were iron-doped zinc oxide (FZ NPs) coated with a phospholipid shell (FZ-3C) and functionalized with the YSA peptide (FZ-3C-YSA NPs) for active targeting of the EphA2 receptor overexpressed in osteosarcoma cells. The results showed that the lipid coating significantly reduced NP toxicity and enhanced stability, while YSA functionalization improved NP internalization in the spheroids. Confocal microscopy revealed broad distribution and internalization of FZ NPs, whereas the lipid-coated FZ-3C NPs showed reduced penetration. However, YSA-functionalized NPs (FZ-3C-YSA) achieved higher internalization and deeper spheroid penetration, particularly when combined with ultrasound stimulation, resulting in significant cytotoxic effects (Carofiglio et al. 2024).

Huang et al. (2012) compared NP uptake and penetration across 2D, 3D, and in vivo models using ultrasmall gold NPs (AuNPs) in MCF-7 breast cancer cells. In monolayers, 2 nm AuNPs had higher cellular uptake than 6 nm and 15 nm AuNPs. In spheroids, penetration of 2 nm and 6 nm AuNPs was time-dependent, with smaller sizes favoring deeper penetration. Pharmacokinetic and biodistribution studies in tumor-bearing mice revealed that smaller AuNPs exhibited slower blood clearance and higher accumulation, while larger AuNPs (15 nm) were predominantly sequestered in the spleen and liver. In all three models, smaller AuNPs (2–6 nm) were partially localized in the cell nuclei, whereas larger NPs (15 nm) remained confined to the cytoplasm (Huang et al. 2012).

Multicellular spheroids can also be used in the study of chemopreventive systems. Salata et al. (2021) evaluated a microemulsion containing fenretinide for its chemopreventive potential in T-47D breast cancer spheroids. Cells were treated with the formulation on day zero, before spheroid formation, and the researchers observed a significant reduction in both spheroid formation efficiency and growth rate. Importantly, these effects were not linked to cytotoxicity, as the treatment was applied at an IC₁₅ concentration (inhibitory concentration for 15% of cells) (Salata et al. 2021). Additionally, tumor spheroids can be employed in the development of more complex in vitro models. For instance, Albanese et al. (2013) incorporated MDA-MB-435 (melanoma) cell spheroids into microfluidic devices to create a ToC platform. Using gold NPs (AuNPs), they assessed the effects of NP size, flow conditions, and receptor targeting NP accumulation. Smaller NPs (40 and 70 nm) presented higher accumulation in spheroids than larger NPs (110 and 150 nm), which compared to in vivo results with the same cell type in xenografted mice. Functionalization of the AuNPs with transferrin also accumulated more in the spheroids, but this was not observed for the in vivo tumors. The authors concluded that the interstitial flow affects NP accumulation by modulating the number of NPs at the tumor interface, with faster flow rates increasing the NP concentration at the interface, leading to greater diffusion and accumulation in the tumor (Albanese et al. 2013).

Heterotypic Tumor spheroids—co-culture of multiple cell lines

While tumor spheroids composed of a single cell line can mimic cell–cell and cell–matrix interactions, as well as oxygen and nutrient gradients, they remain suboptimal models for fully replicating the complexity of the TME. Heterotypic tumor spheroids are one step closer to realistic models. They are produced with multiple cell lines, including at least one tumor cell line. Figure 1C illustrates a spheroid produced with two cell lines, its external structure, and its core.

The incorporation of human cells into heterotypic spheroids enhances their relevance as TME models. Priwitaningrum et al. (2016) developed human tumor spheroids using MDA-MB-231 breast cancer cells with BJ-hTert fibroblasts, and Panc-1 pancreatic cancer cells with human pancreatic stellate cells (hPSCs). These heterotypic spheroids exhibited higher collagen levels compared to monotypic spheroids, effectively mimicking the ECM found in vivo and reducing the penetration of silica and PLGA NPs (Priwitaningrum et al. 2016). This trend was also observed in another study by Priwitaningrum et al. (2023), where increased collagen content in heterotypic spheroids (combining 3T3 mouse fibroblasts and 4T1 mouse breast cancer cells) created a diffusion barrier, leading to lower NP penetration and reduced cytotoxicity of paclitaxel-loaded NPs compared to monotypic spheroids (Priwitaningrum et al. 2023). Consistent with these findings, Roovers et al. (2019) demonstrated that fibroblasts significantly increased type I collagen levels in spheroids. Treating both mono- and heterotypic spheroids with doxorubicin-loaded microbubbles and ultrasound revealed that the denser ECM in heterotypic spheroids conferred greater resistance to cytotoxic treatments (Roovers et al. 2019), further highlighting the importance of stromal components in influencing drug delivery and therapeutic outcomes.

An increased deposition of collagen was also reported by Estrada et al. (2016) in MCF-7 spheroids co-cultured with human dermal fibroblasts. Collagen content significantly increased between days 5 and 15, suggesting active secretion of collagen by the fibroblasts, as this increase was not observed in spheroids composed only of MCF-7 cells. Additionally, co-cultured spheroids presented morphological and phenotypical changes, including altered estrogen receptor and E-cadherin expression, linked to breast cancer aggressiveness, alongside elevated cytokine production and angiogenesis rates compared to monocultures (Estrada et al. 2016).

Bauleth-Ramos et al. (2020) produced heterotypic spheroids composed of HCT-116 colorectal cancer cells, human intestinal fibroblasts (HIFs) and primary monocytes to evaluate Nutlin-3a (Nut3a)-loaded polymeric NPs. HIFs incorporation increased fibronectin expression, indicating augmented ECM production. Although the NPs penetrated poorly into the spheroids, Nut3a-loaded NPs reduced cell viability more effectively than free Nut3a, with up to 78% reduction after 48 h. Co-loaded NPs (Nut3a and GM-CSF) further promoted macrophage polarization towards a pro-inflammatory, anti-tumor M1-like phenotype (Bauleth-Ramos et al. 2020). In a similar approach, Ahvaraki et al. (2024) developed a microfluidic system with heterotypic breast cancer spheroids containing MDA-MB-231 breast cancer cells and human uterine fibroblasts. These spheroids presented an increased ECM compared to monotypic MDA-MB-231 spheroids. While liposomal doxorubicin had similar effects in monolayers, MDA-MB-231 cells in spheroids demonstrated greater resistance to treatment, with heterotypic spheroids being more resistant than monotypic spheroids (Ahvaraki et al. 2024).

Overall, research using spheroids to evaluate NP behavior in tumors demonstrates the influence of NP characteristics—such as size, surface charge, and matrix fluidity—on their penetration into spheroids. These characteristics affect cellular internalization, interactions with the ECM, and the ability of NPs to deform and pass through ECM pores. The frequent differences in treatment responses between 2 and 3D models emphasize the importance of incorporating 3D models into nanomedicine research. Heterotypic spheroids offer a more accurate and reliable model for studying NP behavior, as they better replicate the complex TME, including the formation of a structured ECM. Despite their advantages, spheroid models have limitations, such as incomplete cellular compartmentalization that may not fully mimic in vivo spatial organization and challenges in optimizing culture conditions for multiple cell types (Achilli et al. 2012).

Tumor organoids (tumoroids)

The term “organoid” refers to 3D structures obtained with adult or embryonic stem cells that self-organize to replicate the architecture and their functionality in the tissue of origin. These structures are patient-specific, reflecting the histopathological, genetic, and phenotypic characteristics (LeSavage et al. 2022). Organoids mimic key cellular behaviors, including interactions with the surrounding ECM, making them superior to traditional 2D and regular multicellular spheroid cell cultures as disease models (Sakalem et al. 2021). Tumoroids, in contrast, are a specific subtype of organoids derived from patient tumor tissues. Unlike conventional organoids, which can originate from both normal and diseased tissues, tumoroids exclusively represent malignant tumors, preserving their genetic mutations, cellular heterogeneity, and TME components, including cancer-associated fibroblasts and immune cells (Barbáchano et al. 2021). Tumoroids, as patient-derived stroma/tumor organoids, have been increasingly recognized as models in precision medicine due to their ability to replicate the characteristics of individual tumors and their microenvironment (Neal et al. 2018; Sachs et al. 2018). By integrating these models with high-throughput screening and multi-omics technologies, researchers can evaluate drug sensitivity and resistance mechanisms at a patient-specific level. This makes tumoroids beneficial for personalized cancer therapy, enabling tailored treatment strategies that improve clinical outcomes (Xu et al. 2022). This review focuses on organoids derived from tumor tissues (tumoroids), and Fig. 2 depicts an organoid composed of 4 types of cells to form a complex system that replicates tissue architecture and cellular interactions.

Fig. 2

Structural differences between organoids and tumor-on-a-chip—nanocarriers are represented as spheres within the models. Organoids are 3D cell culture systems derived from one or more patient-derived cells or stem cells, including, for example, goblet, paneth, enterocytes, neural, basal, luminal, and tumor cells (illustrated in various shapes and colors in the figure). Organoids replicate specific structural and functional features of tumors, including cellular heterogeneity, spatial organization, and self-organization capacity, within a supportive extracellular matrix. However, organoids lack the dynamic fluid flow, mechanical forces, and controlled environmental conditions provided by tumor-on-a-chip systems. In contrast, the tumor-on-a-chip is a microfluidic platform that integrates mechanical, biochemical, and fluidic cues to more accurately mimic the tumor microenvironment. This system features interconnected channels and compartments that enable the co-culture of multiple cell types, real-time monitoring of cellular responses, and precise manipulation of factors such as nutrient gradients and oxygen levels. The schematic design of the chip in the figure is representative and serves as an example of a 3-channel ToC organized to allow communication between the channels; in practice, these systems can be fabricated in various designs with many different cells, such as endothelial cells, fibroblasts, mesenchymal stem cells, immune cells and tumor cells, to meet specific experimental needs

The production of tumoroids varies significantly due to differences in cell sourcing and collection methods. They can be derived from primary tumors, metastatic lesions, circulating tumor cells, or tumor cells from liquid effusions. These materials are collected through solid and liquid biopsies, surgical resections, or autopsies (Gao et al. 2014; Vlachogiannis et al. 2018; LeSavage et al. 2022). Tumor tissue is then processed for disassociation into small fragments, cell clusters, or single cells using mechanical disruption and/or enzymatic digestion, and cells are subsequently cultured under conditions promoting stem cell growth. Tumoroids have been developed using this methodology to create living biobanks, enabling large-scale drug screenings and molecular analysis (Van de Wetering et al. 2015; Veninga and Voest 2021).

Organoids can also be cultured at the air–liquid interface using collagen as scaffold. Li et al. (2014) demonstrated that this platform supports primary organoid generation and facilitates oncogenic transformation. By providing improved access to nutrients and oxygen, this method promotes cell differentiation and growth. Additionally, advancements in CRISPR-CAS9 technology have enabled precise genetic modifications in organoids, such as the introduction of driver gene mutations, for studying cancer initiation, progression, and molecular mechanisms. The integration of CRISPR-CAS9 with organoid models offers a flexible approach to simulate tumor genetic diversity, enhancing the accuracy of cancer research (Wood and Ewald 2021; LeSavage et al. 2022).

Tumoroids became important due to their capacity to represent cancer heterogenicity and the complex interactions within the TME in vitro. Patient-derived organoids retain immune cells (e.g., natural killer (NK) cells, T and B lymphocytes, and macrophages) as well as stromal cells, mimicking the endogenous TME. Neal et al. (2018) demonstrated that organoids can preserve native immune cell lines, including cancer-associated fibroblasts, NK cells, and lymphocytes, for personalized immunotherapy testing. This indicates that patient-derived organoids provide a more complex and native TME compared to other 3D models (Neal et al. 2018).

The organoid platform offers multiple experimental applications, including single-cell transcriptomics, gene edition, and tagging, xenotransplantation, and co-culture with non-tumor cells such as cancer-associated fibroblast and immune cell types (Kretzschmar 1990; LeSavage et al. 2022). Organoids can also be co-cultured with autologous immune cells isolated from the patient’s peripheral blood, lymph nodes, or tumors, enabling researchers to study cancer cell interactions within their microenvironment, providing a deeper understanding of how stromal and immune cells influence tumor behavior. These systems allow the exploration of critical processes such as invasion, metastasis, progression, and tumorigenesis, facilitating the discovery of novel therapeutic strategies (Wood and Ewald 2021; Xu et al. 2022).

In cancer research, tumoroids are mostly used to model carcinomas, including colorectal, prostate, bladder, breast, kidney, liver, and pancreas (Gao et al. 2014; Van de Wetering et al. 2015; Sachs et al. 2018; Seino et al. 2018; Mullenders et al. 2019; Calandrini et al. 2020; Dong et al. 2022). Deng et al. (2022) developed a colorectal cancer tumoroid model to evaluate the effects of doxorubicin-loaded nanoclusters. Similar to what is observed when comparing monolayers and 3D models, they noted that SW480 cells were more sensitive to the nanoclusters than the tumoroid models. Moreover, they observed a dose-dependent effect of the NPs, with increasing concentrations causing progressive damage to the tumoroid structure (Deng et al. 2022).

Another example of tumoroids employed as predictive platforms for nanomedicine is the robust study conducted by Obaid et al. (2019) that developed a photo-immunonanoconjugate containing a benzoporphyrin derivative conjugated to an anti-EGFR monoclonal antibody for the treatment of pancreatic ductal adenocarcinoma. To assess the efficacy of this nanoconjugate, the researchers employed monolayer cultures, patient-derived tumor organoids composed of MIA PaCa-2 cancer cells and pancreatic cancer-associated fibroblasts, and in vivo heterotypic murine xenograft model. The study demonstrated that the organoid model effectively recapitulated the TME, allowing for the assessment of the nanoconjugate’s binding specificity, penetration, and photodynamic efficacy. The findings from the organoid model closely reflected the treatment responses observed in the in vivo xenograft model, such as the significantly enhanced antitumor efficacy and the ability of the nanoconjugate to penetrate deep into the tumor stroma (Obaid et al. 2019).

Likewise, Camorani et al. (2024) also employed a organoid model to evaluate NPs antitumoral effect. The authors developed a multifunctional nanoplatform base on gold-core/silica-shell NPs embedded with a photosensitive and luminescent iridium(III) complex, designed to target EGFR and/or PDGFRβ positive tumors. To evaluate the antitumor efficacy of the NPs, the authors employed patient-derived organoids composed of breast cancer cells, vascular endothelial cells, and mesenchymal stromal cells. They demonstrated that the nanoplatform effectively penetrated the organoid mass and, upon light irradiation, the NPs exerted potent phototherapeutic cytotoxicity, reducing cell viability by more than 70% compared to untreated controls (Camorani et al. 2024).

However, the establishment of organoids and tumoroids faces challenges, including high costs of specialized culture media containing complex growth factor combinations, variability in success rates depending on tumor type and tissue quality, and competition from normal cells outgrowing tumor cells. Additionally, the absence of certain TME components, such as immune cells, limits studies of tumor-immune interactions. Experimental conditions must be carefully controlled, as some media components may interfere with drugs targeting similar pathways (Van de Wetering et al. 2015; Veninga and Voest 2021). Despite these challenges, organoids and tumoroids provide a highly versatile, patient-specific platform that bridges the gap between traditional cell cultures and in vivo models, making them an indispensable tool for translational cancer research. Their ability to closely mimic the complexity of human tissues allows for more accurate modeling of disease processes and personalized therapeutic responses.

Tumor-on-a-chip (ToC) models

The ToC models integrate microfluidic technology, 3D cell culture, and tissue engineering, offering an advanced platform to replicate the physicochemical properties of the TME. This technology enables control over factors that influence tumor behavior, such as the composition of the ECM, application of mechanical forces, and manipulation of oxygen levels (Liu et al. 2021b). By integrating these elements, ToC models provide reliability and a more accurate and representative system for studying interactions and anti-tumor effects within the TME. Microfluidic chips equipped with integrated gas pressure micro-pistons have been developed to mimic compressive stress, which influences various cancer-related processes (Liu et al. 2020a). Pistons embedded directly into microfluidic chips use pressurized gas to push a tiny membrane, creating a compressive force on the chamber containing cancer cells (Onal et al. 2021). To control interstitial fluid pressure, syringes, and peristaltic pumps are employed (Aleman and Skardal 2019; Zheng et al. 2021). ToC technology offers strategies to control oxygen levels, chemical reactions that consume or generate oxygen, and cell consumption of oxygen (Chang et al. 2014; Palacio-Castañeda et al. 2020; Zheng et al. 2021), and to simulate the complex cellular composition and architecture of TME. The TME in these systems is characterized by distinct oxygen gradients, with hypoxic zones developing due to limited oxygen diffusion and the high metabolic demands of proliferating cancer cells (Harris 2002).

Most models employ immortalized cell lines for their ease of use and culture. However, these cell lines do not fully represent in vivo tumors, hindering the translation of in vitro studies to clinical applications. To address this limitation, researchers are increasingly employing primary cells in ToC designs, providing a more realistic representation of the TME (Son et al. 2017; Bouquerel et al. 2023). Additionally, some studies have incorporated human induced pluripotent stem cells (iPSCs), which can differentiate into various tissue types, allowing for multi-organ platforms from the same donor and the study of complex tumor-organ interactions (Burridge et al. 2016; Kurokawa et al. 2017), organoids (Jung et al. 2019; Hwangbo et al. 2024), patient-derived xenografts (Ong et al. 2020), and fresh surgical tumor samples (Hattersley et al. 2012; Dorrigiv et al. 2021). While these models offer the highest level of biological fidelity, their implementation can be challenging due to factors such as inter-patient variability and limited long-term viability. Figure 2 depicts the structural differences between an organoid and a ToC.

ToC models can recreate cellular architecture through various microfabrication designs, which include: (i) compartmentalized chips with distinct chambers for housing various cell types (Azadi et al. 2021); (ii) micro-well arrays for co-culture of various cell types in close proximity (Azizipour et al. 2022); (iii) membrane chips, which use porous membranes to separate compartments, allowing studies on cell migration and paracrine signaling (Choi et al. 2015); (iv) lumen chips that incorporate hollow channels to mimic blood vessels, facilitating investigations of vascularization, drug delivery, and cancer cell extravasation (Pradhan et al. 2018). The field of ToC technology is constantly evolving, and many configurations combine elements from these categories (Sleeboom et al. 2018).

Various ToC models have been developed, including those for breast, lung, colorectal, brain and pancreatic tumor (Kashaninejad et al. 2016; Neal et al. 2018; Liu et al. 2021b). Breast tumor chips incorporate microfluidic channels resembling mammary ducts and blood vessels (Cauli et al. 2023), and a 3D bone chip model to study bone metastasis, a common site of breast cancer spread (Hao et al. 2018). Lung tumor chips mimic lung physiological functions by incorporating human lung epithelial cells and pulmonary microvascular endothelial cells separated by a porous ECM, replicating the air-blood barrier and mucociliary clearance to study tumor cell growth under simulated breathing movements (Ruzycka et al. 2019; Khalid et al. 2020).

Liu et al. (2015) developed a ToC model using U251 human glioma cells, integrating microvascular systems to investigate the effects of folate-decorated polymeric NPs loaded with coumarin and paclitaxel. This microfluidic platform enabled real-time monitoring of NP accumulation and therapeutic response within a controlled 3D TME. By employing live-dead fluorescence microscopy, the authors observed that paclitaxel-loaded folate-functionalized NPs exhibited significantly higher accumulation in tumor cells compared to folate-free NPs, confirming the active targeting effect. Moreover, the folate-NP-treated group demonstrated increased cytotoxicity, with a higher proportion of dead cells compared to the non-functionalized NP group (Liu et al 2015).

A colorectal ToC model developed by Carvalho et al. (2019) features a central chamber containing HCT-116 colorectal cancer cells flanked by two perfusable channels lined with human colonic microvascular endothelial cells (HCoMECs). The lumen-like channels mimic microvascular function, supplying nutrients to the surrounding tumor tissues and ensuring their growth. The authors utilized this platform to validate the efficacy of gemcitabine-loaded dendrimers in a gradient fashion. They observed that the fluorescent dendrimers were homogeneously perfused throughout the tumor core, enabling real-time monitoring of drug distribution and treatment response. Live/dead fluorescence microscopy analysis revealed an increase in cell death in tumor regions exposed to NPs, achieving a 40-fold higher cytotoxic effect compared to untreated controls, demonstrating that the dendrimers effectively delivered the drug and promoted an anti-tumor effect (Carvalho et al. 2019).

Another interesting study was conducted by Wang et al. (2018), using a multicompartment, vascularized ToC model with human umbilical vein endothelial cells (HUVECs) and human ovarian cancer cells (SKOV3) to investigate the EPR effect of PEG-liposome and PEG-PLGA NPs on the tumor. The highlight of this study was that the results were comparable to the in vivo experiments with BALB/c nude female mouse xenograft model. While in 2D and 3D models, they have observed that a modification to the NPs with folic acid promoted higher cellular uptake due to active targeting, this enhancement was nonsignificant in the ToC and in vivo tumor models. Furthermore, in both ToC and in vivo models, the NPs persisted in the tumor for hours to days. Their long-term interactions with tumor tissue were comparable, concluding that the ToC model is physiologically closer to in vivo models and an effective system to study the transport efficacy of NPs-based drug delivery platforms (Wang et al. 2018).

Additionally, innovative multi-organ platforms have been developed to simulate tumor metastasis, such as the chip designed by Sharifi et al. (2020) that mimics hepatocellular carcinoma–bone metastasis. Using this model with HepG2 fluorescent cells and a bone-mimetic compartment containing hydroxyapatite, the authors evaluated the anti-tumor effect of a thymoquinone-loaded polymeric NP. They reported a reduction of approximately twofold in cell density and metastasis towards the bone compartment in the NP group when compared to the free drug and attributed the effect to the controlled release of the drug from the NP formulation (Sharifi et al. 2020).

ToC platforms are compatible with various analytical methods to assess the biological effects of NPs. Cellular viability assays (e.g., live/dead and LDH assays) provide initial information on cytotoxicity and potential therapeutic effects (Carvalho et al. 2019; Mitxelena-Iribarren et al. 2019; Khot et al. 2020). Microscopy techniques (e.g., fluorescence and confocal) allow visualization of NPs localization, uptake, and intracellular distribution (Albanese et al. 2013; Xu et al. 2016, Carvalho et al. 2019). Lumen and membrane chips can be tailored to study specific processes like angiogenesis and cell migration (Choi et al. 2015; Pradhan et al. 2018; Liu et al. 2021b). They are also compatible with gene and protein expression analysis techniques, such as quantitative PCR (qPCR) and western blotting, and can integrate omics analyses to study molecular pathways (Jellali et al. 2021). Techniques like RNA sequencing (RNA-seq) provide information into transcriptomic profiles (Conesa et al. 2016), while mass spectrometry facilitates the identification and quantification of a wide range of proteins within TME (Shuken 2023). Moreover, biosensors integrated within ToC platforms offer real-time monitoring of cellular responses to NPs, measuring factors such as pH, oxygen levels, and specific biomarkers to assess cellular metabolism and function (Kashaninejad et al. 2016; Khalid et al. 2020). ToC technology even extends to studying stimuli-responsive NPs by incorporating nano-photosensitizers activated by light within tumor cells to explore the therapeutic potential of photodynamic therapy (Chudy et al. 2018).

Microfluidic-based ToC models offer significant advantages over static culture models by incorporating fluid flow, which better mimics the dynamic conditions of the TME. The presence of controlled flow enhances nutrient and oxygen exchange, reduces waste accumulation, and replicates the interstitial fluid dynamics observed in vivo, providing a more physiologically relevant environment for studying NP behavior (Aleman and Skardal 2019; Zheng et al. 2021). This dynamic flow is of great importance for NP-based drug delivery studies, as it influences NP transport, extravasation, and diffusion within tumor tissues (Ruzycka et al. 2019). Additionally, microfluidic systems allow for precise manipulation of shear stress and oxygen gradients, which impact tumor cell behavior, drug resistance, and NP uptake (Kashaninejad et al. 2016; Chang et al. 2014; Zheng et al. 2021). However, the operational complexity of microfluidic ToC models, including the need for specialized equipment, precise flow control, and technical expertise, can pose challenges such as variability in flow conditions, potential clogging of microchannels, difficulties in long-term culture maintenance, and higher costs associated with fabrication and operation (Battat et al. 2022). In contrast, static 3D models, while simpler, more accessible, and easier to scale, lack fluid dynamics, mechanical forces, and controlled microenvironmental gradients, limiting their ability to fully replicate the nutrient diffusion, interstitial flow, and shear stress found in vivo.

Key aspects of in vitro models

While 2D models are still widely used for initial screening, the field is increasingly moving towards 3D models, particularly heterotypic spheroids, organoids, and ToC systems, to better predict clinical outcomes. This evolution of in vitro cancer models has enhanced our ability to replicate the complexity of the TME. While 2D models remain valuable for preliminary screening due to their simplicity and cost-effectiveness, they lack the physiological relevance of 3D models, which better mimic cell–cell interactions, nutrient gradients, and drug penetration dynamics.

Compared to spheroids composed of single cell lines, heterotypic tumor spheroids offer a more accurate representation of the TME due to the incorporation of multiple cell types, such as cancer-associated fibroblasts and immune cells, which influence drug resistance and therapeutic outcomes. These models are particularly useful for studying drug penetration and resistance mechanisms. On the other hand, tumor organoids, derived from patient-specific tumor tissues (tumoroids), provide a personalized approach to cancer research, preserving genetic and phenotypic heterogeneity. However, their high cost and variability in culture success rates remain challenges. While, ToC models, integrating microfluidic technology, offer dynamic control over the TME, including fluid flow, oxygen gradients, and mechanical forces, making them highly relevant for studying NP behavior in a more physiologically accurate setting. These systems allow precise manipulation of biomechanical and biochemical conditions, further improving model accuracy. Despite their advantages, ToC platforms require specialized equipment and expertise, limiting their widespread adoption.

Methodological approaches

After reviewing the potential in vitro models for studying NP delivery in cancer, this section will explore the main methodologies used to assess the performance and efficacy of nanocarriers. Such methodologies encompass the investigation of cytotoxicity, cellular uptake, and therapeutic efficacy of NPs in cancer treatment. The following subsections will explore how these methods are designed to answer specific experimental questions, their underlying principles, and their applicability across the various in vitro cancer models, highlighting their advantages and limitations to provide an overview of the tools available to investigate NP-based cancer therapies. Figure 3 depicts the structure of five of the most-cited NPs in this article.

Fig. 3

Illustration of various nanoparticle formulations for drug delivery. A Metallic nanoparticle—inorganic nanoparticles composed of metallic elements; B Polymeric nanoparticle—composed of biocompatible and biodegradable polymers; C Liposome—a vesicular system made of a phospholipid bilayer with an aqueous core; D Nanoemulsion oil-in-water (O/W)– A colloidal dispersion of oil droplets stabilized by a surfactant layer. Though nanoemulsions are lipid-based systems, their structure differs significantly from lipid nanoparticles as they lack a solid or semi-solid lipid matrix. E Lipid Nanoparticle (LNP)—A modern lipid-based nanocarrier designed primarily for nucleic acid delivery (e.g., mRNA). Unlike Solid Lipid Nanoparticles (SLNs), composed of solid lipids, and Nanostructured Lipid Carriers (NLCs), which combine solid and liquid lipids, LNPs may include ionizable lipids, cholesterol, and PEG-lipids. These components enable efficient encapsulation and delivery of genetic materials

Nanoparticle uptake evaluation

Understanding the mechanisms of cellular uptake and intracellular trafficking of nanomaterials is critical for the comprehension of how NPs reach their site of action and exert their cytotoxic effects on cancer cells. Optimal NP design can favor a specific cellular internalization pathway that, in turn, will affect the intracellular trafficking of the NPs and encapsulated molecules (Behzadi et al. 2017). This is extremely important in cancer therapy, as many targets of cytotoxic drugs are located in subcellular compartments, so their intracellular distribution is an essential factor for the therapy efficacy (Rennick et al. 2021). Thus, uptake studies can guide the design of NPs that are selectively taken up by cancer cells over healthy cells (Cong et al. 2021).

A considerable research effort has been done in this field since the uptake of nanomaterials is quite different from small-molecule drugs and large macromolecules (Iversen et al. 2011; Mosquera et al. 2018; Rennick et al. 2021). While small molecules undergo passive diffusion and macromolecules rely on endosomal escape to reach the cytoplasm, NP uptake is a complex process and depends on several properties, such as size, shape, surface charge, surface functional groups, and particle hydrophilicity (Sahay et al. 2010; Kettler et al. 2014; Sohrabi Kashani and Packirisamy 2021). However, despite the complexity of this topic, until today, literature basically relies on the use of fluorescent labeling materials, uptake inhibitors and monolayer cell culture models, and more complex models are still missing (Rennick et al. 2021).

Among the different cellular uptake mechanisms, five are of great relevance for the internalization of nanomaterials: (i) clathrin-dependent endocytosis; (ii) caveolin-dependent endocytosis; (iii) clathrin- and caveolin-independent endocytosis; (iv) phagocytosis; and (v) micropinocytosis (Iversen et al. 2011; Gong et al. 2019). These different pathways are typically studied in vitro by fluorescence-based assays, in which cells are seeded in monolayers and treated with fluorescent-labeled NPs and endocytosis markers/inhibitors, and uptake levels are evaluated by fluorescence imaging or flow cytometry (Drasler et al. 2017; Rennick et al. 2021). For example, in order to identify the endocytic pathways of Boltorn® H30 polyester NPs, Zeng et al. (2012) incubated MDA-MB-468 cells with fluorescent-labeled NPs and endocytosis dyes prior to imaging using confocal microscopy; dextran-rhodamine conjugates, Alexa 647-labeled transferrin and Alexa 647-labeled cholera toxin B subunit were used as markers of macropinocytosis, clathrin-mediated and caveolin-mediated endocytosis, respectively (Zeng et al. 2012). High colocalization was observed with transferrin and dextran, suggesting that clathrin-mediated endocytosis and macropinocytosis were the most significant uptake mechanisms for the developed NP.

The imaging techniques, although simple and highly sensitive, will only provide a semi-quantitative result compared to a control since the fluorescence signal will depend on the source of excitation, the number of markers incorporated in the NP, and the sensitivity of the detector (Rennick et al. 2021). In addition, it is difficult to conclude whether the NPs have been internalized or just bound to the cell surface (Iversen et al. 2011). Flow cytometry, in turn, captures the fluorescent signal of individual cells, having greater sensitivity. For example, Dos Santos et al. (2011) investigated carboxylated polystyrene NPs uptake in different cancer models using endocytosis inhibitors and flow cytometry. Genistein, chlorpromazine, nocodazole and cytochalasin were selected for the inhibition of caveolae-mediated endocytosis, clathrin-mediated endocytosis, microtubule cytoskeleton and actin polymerization (an index of macropinocytosis), respectively. Human glial astrocytoma 1321N1, lung epithelium A549 and cervix epithelium HeLa cells were incubated with fluorescently labeled NPs (40 or 200 nm) and endocytosis inhibitors, and levels of inhibition of uptake were evaluated. While NP uptake was mainly inhibited by cytochalasin in HeLa cells, chlorpromazine presented a higher level of NP uptake inhibition in 1321N1 and A549 cell lines, suggesting that clathrin might be involved in NP internalization. Additionally, stronger inhibition effects were observed for larger NPs. This suggests that the inhibition of endocytosis is dependent on the type of cell and on the properties of the studied NP (Dos Santos et al. 2011).

Despite being a widely used technique, the use of endocytosis pathway inhibitors to assess the uptake of NPs has been recently questioned, since different endocytic pathways may share protein components, reducing the specificity of this assay. To overcome this limitation, recent studies have focused on genetic inhibition of endocytosis with siRNA, aiming to reduce off-target effects and to obtain a more robust tool to study the role of different endocytic pathways in the uptake of NPs (Rennick et al. 2021). Considering the promising of lipid NPs mediated delivery of siRNA in cancer therapy, Gilleron et al. (2013) studied NP uptake in vitro aiming to elucidate the precise molecular mechanisms of cargo delivery to cancer cells. HeLa cells were transfected with siRNA to downregulate key components of different endocytic pathways prior to treatment with fluorescently labeled NPs and fluorescence imaging. A 50–70% reduction of NP uptake was observed upon downregulating clathrin heavy chain and macropinocytosis genes, suggesting that both pathways are required for NPs internalization (Gilleron et al. 2013).

The 3D cell culture models have recently been developed to understand the internalization mechanisms of nanomaterials better, offering a more physiologically relevant representation of the in vivo cellular microenvironment compared to monolayer cultures. Similar to monolayer models, techniques such as flow cytometry and confocal microscopy, along with inhibitors of key proteins of each endocytic pathway, are employed to study NP uptake (Verdera et al. 2017). Typically, spheroids are cultured in 96-well plates, treated with endocytosis inhibitors for 48 h, and subsequently washed with PBS and an acid wash buffer to eliminate membrane-bound NPs before analysis. Importantly, distinguishing between internalization and surface adsorption is crucial for accurately interpreting results, as internalized NPs are transported into the cytoplasm or subcellular compartments, while adsorbed NPs remain bound to the cell membrane. This distinction is particularly important in 3D models, where the complex architecture and ECM can lead to significant NP adsorption without actual internalization. To differentiate between these processes, researchers often employ acid wash buffers to remove surface-bound NPs before analysis (Patel et al. 2019). For instance, Verdera et al. (2017) demonstrated the use of a low-pH buffer to remove membrane-bound NPs from spheroids, ensuring that only internalized NPs were quantified via flow cytometry or confocal microscopy (Verdera et al. 2017). Additionally, advanced imaging techniques, such as z-stack confocal microscopy, can provide spatial resolution to confirm NP localization within cells versus surface adsorption (Gilleron et al. 2013). Despite these advancements, the efficacy of this approach may be limited by challenges in imaging, such as the difficulty of achieving high-resolution visualization in dense 3D spheroid structures, as well as the lack of specificity of certain endocytosis inhibitors, which can lead to off-target effects and ambiguous interpretations of NP uptake mechanisms in 3D structures.

Flow cytometry is commonly used to evaluate NP interactions with spheroids/organoids, offering a quantitative and high-throughput approach to assess uptake and penetration into the 3D structure. After dissociating spheroids into single-cell suspensions, flow cytometry can quantify the fluorescence intensity of cells treated with fluorescently labeled NPs, to analyze the extent of internalization. This method can be useful, for example, for comparing the uptake efficiency of NPs with different physicochemical properties or surface modifications (Verdera et al. 2017; Tchoryk et al. 2019). Furthermore, this technique is particularly advantageous when used alongside fluorescence imaging, as it allows for direct quantification of uptake while mitigating the limitations associated with imaging depth in dense spheroid structures (Verdera et al. 2017).

In addition to understanding cellular uptake mechanisms, elucidating how NPs penetrate 3D structures, such as spheroids and organotypic constructs, is crucial. NPs can penetrate tissues through various mechanisms, including paracellular transport (between cells), transcellular transport (through cells), or passive diffusion through the ECM. Transcellular transport involves NPs being internalized by a cell and subsequently released into the ECM or neighboring cells, while paracellular transport occurs when NPs diffuse between adjacent cells without direct internalization (McCright et al. 2023). To study penetration routes, advanced imaging techniques, such as multiphoton microscopy (Yamada et al. 2014) and light-sheet fluorescence microscopy (Stelzer et al. 2021) have been employed, which provide deeper tissue penetration and higher resolution in 3D models compared to traditional confocal microscopy. Additionally, the use of markers that selectively label tight junctions or ECM components can help visualize NP interactions with these structures, providing information about their penetration routes (Chen et al. 2024; López-Méndez et al. 2024). Understanding these penetration mechanisms is specifically relevant in cancer therapy, as enhanced NP diffusion within tumor 3D structure can improve therapeutic efficacy by reaching deeply located cancer cells that may otherwise be inaccessible.

Cell viability assessment

Among the in vitro methodologies used in cancer models, the viability assays represent one of the most employed. In the context of NP–mediated delivery and cancer, it is possible to evaluate the cytotoxicity of the nanocarriers as well as the influence of drug encapsulation/association on cytotoxicity, which is indicated by cell survival or death upon exposure to a wide concentration range of the test agent. Various methods are used to assess cell viability, each with its own advantages and limitations (Adan et al. 2016; Khalef et al. 2024). Table 2 summarizes the main techniques available.

Table 2 Overview of techniques for cell viability assessment in 2D and 3D cultures: applicability, limitations, and optimization strategies

Compared to 2D monolayers, 3D models require methodological optimizations to ensure accurate NP cytotoxicity measurements. Several strategies can be employed to improve the reliability of cytotoxicity assessments in 3D models, depending on their structural characteristics. In the case of spheroids or organoids with varying sizes, pre-selecting spheroids/organoids based on volume and shape uniformity using automated image analysis helps to minimize variability (Zanoni et al. 2016). For dense 3D structures requiring dissociation prior to cytotoxicity measurements, protocols employing gentler dissociation methods can be utilized to minimize cellular damage and maintain cell viability (Pleguezuelos-Manzano et al. 2020). Alternatively, for 3D structures where dissociation is not required, protocol optimizations can include extended incubation times to ensure uniform reagent penetration throughout the entire 3D structure (Sambale et al. 2015) and the implementation of confocal Z-stack image acquisition, allowing for detailed volumetric reconstructions of the 3D model to assess NP-induced cytotoxicity more accurately (Kim et al. 2020). In the following section, we will explore protocol optimizations in 3D models that employ one or more of these approaches to enhance the reproducibility of cytotoxicity assessments.

Metabolism-based reagents

Metabolism-based reagents assess cell viability by measuring the metabolic activity of living cells, generating colorimetric, fluorescent, or luminescent products that can be quantified. Among these, the MTT assay is widely used to evaluate mitochondrial metabolic activity by measuring the conversion of 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide into formazan crystals by mitochondrial oxidoreductase enzymes in living cells. Since its introduction by Mosmann (1983), the MTT assay has been regarded as a gold standard for colorimetric cell viability assessment. In the context of NPs for cancer treatment, the MTT assay is widely used to compare the cytotoxicity of free-drugs and drug-loaded NPs. For example, Kilicay et al. (2024) demonstrated that co-encapsulated caffeic acid and folic acid NPs selectively reduced MCF-7 breast cancer cell viability, while Barbosa et al. (2024) showed that niclosamide-loaded nanoemulsions enhanced cytotoxicity in HCT-116 colorectal cancer cells compared to the free drug.

Despite its reliability, the MTT has notable limitations, including interference from cell density, incubation time, and nanoformulation components. Some nanocarriers, such as metallic NPs (Mello et al. 2020; Ghasemi et al. 2021) and liposomes (Angius and Floris 2015), can chemically or optically interact with the assay, leading to over- or underestimation of viability (Carvalho et al. 2017; Mello et al. 2020). To further exemplify, Carvalho et al. (2017) reported a possible interference between the MTT reagent and nanocarriers. In their study, micro and nanoemulsions (ME and NE, respectively) were developed to target skin cancer, and their effects on engineered skin viability were investigated using MTT assay. While MTT results suggested a more pronounced cytotoxicity mediated by NE compared to the ME, histology tests revealed the opposite. These results indicated an underestimation of viability, probably due to the ME composition, since it contained a vitamin E-derived surfactant that may have participated in the reduction of MTT (Carvalho et al. 2017).

The MTT is not well suited for 3D models, as formazan crystals accumulate unevenly in spheroids, restricting penetration into deeper layers. To address these limitations, alternative tetrazolium-based assays, such as MTS, WST-1, and WST-8, have been developed, producing soluble formazan and eliminating the solubilization step. While MTS offers stability for extended incubations, WST-1 and WST-8 offer higher sensitivity, making them more effective for detecting subtle differences in viability (Riss et al. 2013; Oner et al. 2023; Khalef et al. 2024). In this context, Zanoni et al. (2016) employed the WST-1 assay to assess cytotoxicity in 3D tumor spheroids without requiring prior dissociation of the structure. The protocol involved transferring individual spheroids into 96-well plates, followed by the addition of the WST-1 reagent directly to each well. To ensure reliable viability measurements, an extended incubation period of 4 h was applied, allowing sufficient reagent penetration and metabolic conversion in the dense 3D structures. Additionally, the researchers minimized variability by pre-selecting spheroids based on volume and shape uniformity using automated image analysis, which helped standardize metabolic activity within the 3D models. This assay demonstrated low variability, with an average coefficient of variation of 7.53%, demonstrating its suitability for accurately assessing cytotoxicity in spheroid models.

Another metabolism-based reagent widely used is resazurin, which has gained recognition as the Alamar Blue assay since its commercialization by Trek Diagnostics in 1993. It is also marketed under different names such as Vybrant™ (Biotium and Invitrogen) and UptiBlue™ (Interchim, France) (Khalef et al. 2024). This assay is based on the reduction of resazurin into resofurin, a red fluorescent product, representing both a colorimetric and fluorescent method for assessing cell viability. Resazurin’s dual-mode detection (absorbance or fluorescence) provides flexibility in analysis (Adan et al. 2016; Gong et al. 2020). Sambale et al. (2015) conducted a cytotoxicity assessment in 2D monolayers and spheroids (A549 and NIH-3T3 cells), using the resazurin assay, to assess the effects of zinc oxide NP (ZnO-NP) and titanium dioxide NP (TiO₂-NP). Due to diffusion limitations in 3D structures, which are absent in monolayers, an extended incubation period of 20 h was required in spheroids, compared to 2 h in monolayers, ensuring adequate reagent penetration and metabolic conversion. The results showed that ZnO-NP exhibited cytotoxic effects in both 2D and 3D cultures, with A549 cells being more sensitive in 3D spheroids compared to 2D monolayers. Conversely, NIH-3T3 cells showed similar sensitivity in both 2D and 3D cultures. TiO₂​-NP, on the other hand, was non-toxic in 2D cultures but influenced spheroid formation and cell viability in 3D cultures, particularly for A549 cells.

The measurement of ATP levels, another common method to assess cell viability, relies on detecting intracellular ATP through a luminometric reaction. It is highly sensitive and quick, producing results in less than an hour with a straightforward protocol involving only a few steps. All ATP-based viability assays require cell lysis; however, the method of lysis differs between standard assays that require prior dissociation and those specifically optimized for intact 3D structures (Xie and Wu 2016; Dominijanni et al. 2021). Traditional ATP quantification assays, such as the Adenosine 5′-triphosphate (ATP) Bioluminescent Somatic Cell Assay Kit (Sigma-Aldrich) and CellTiter-Glo® Luminescent Cell Viability Assay (Promega Corporation), were originally developed for 2D cultures and, therefore, require enzymatic and/or mechanical dissociation before measurement to ensure ATP extraction from all cells. In contrast, assays specifically optimized for intact 3D structures, such as ATPlite™ 1Step 3D (Revvity, Inc.) and CellTiter-Glo® 3D (Promega Corporation) use enhanced detergent-based lysis buffers that allow ATP release without requiring prior dissociation (Riss et al. 2013). These assays specific for 3D structures ensure reagent penetration throughout spheroids and organoids. In a study by Zanoni et al. (2016), the CellTiter-Glo® 3D assay required a 30-min incubation at room temperature to allow sufficient reagent penetration into spheroids up to 650 μm in diameter, as confirmed by Light Sheet Fluorescence Microscopy imaging. This method exhibited low variability, with an average CV of 7.23%, demonstrating its suitability for accurately assessing cytotoxicity in spheroid models. However, for larger or denser 3D structures, some protocols, such as in Sundar et al. (2022), still incorporate mechanical trituration before ATP measurement with the CellTiter-Glo® 3D assay to further enhance lysis and signal accuracy. In their glioblastoma organoid model, organoids were manually triturated twice, with a 5-min incubation between each trituration step, followed by a 1:4 dilution of lysates in a 96-well plate. The mixture was then incubated for 10 min before luminescence measurement, ensuring effective ATP extraction from all regions of the 3D structure.

Intracellular integration or accumulation-based reagents

Reagents in this category evaluate cell viability based on the ability of living cells to retain or accumulate specific molecules. Among these, sulforhodamine B (SRB) binds to intracellular proteins, indirectly measuring total cell mass. It is an anionic amino xanthene with available sulfonic groups in acid media, which can interact with basic amino acid residues from the trichloroacetic acid used to fix the cells. SRB is highly sensitive and reliable for adherent cells, making it a good choice for 2D cultures (Adan et al. 2016). In 3D systems, however, dissociation of spheroids or organoids is necessary to achieve uniform reagent access and accurate quantification, which can complicate the experimental workflow.

Although less common than MTT, the SRB assay has been employed to assess the cytotoxicity of various types of NPs toward cancer cell lines. Jain et al. (2023) evaluated the effect of silver NPs (AgNPs) containing extracts of distinct species of Curcuma on human colon cancer cell line HT-29. Viability results provided by the SRB methods showed decreased viability after treatment with AgNPs in a dose-dependent manner and also determined which Curcuma species exhibited better activity (Jain et al. 2023). Another study conducted by Mousa et al. (2023) employed SRB to access the cytotoxic effect of zinc oxide NPs (ZnOxNPs) synthesized from plants on human ovarian cancer cell line SKOV3, demonstrating a significant decrease in cell viability at higher concentrations (Mousa et al. 2023). They reported their results to be very distinct from Alipour et al.’s findings, which were obtained using an MTT assay (Alipour et al. 2022).

In addition to cell viability, it is also possible to investigate cell proliferation utilizing SRB assay. For instance, Teixeira et al. (2019) evaluated the impact of poly (lactic-co-glycolic) acid (PLGA) NPs containing 1,3-dihydroxy-2-methylxanthone (DHMXAN) on cell growth of human breast cancer cell line MCF-7, in which the incorporation of DHMXAN in NPs resulted in higher growth inhibition than free-DHMXAN (Teixeira et al. 2019). Relevant drawbacks should be considered when selecting the SRB assay, such as the impossibility of assessing cell functionality since the measurement is based on total protein amount, therefore cell survival or death is not distinguished (Adan et al. 2016). Even though cell fixation enables long-term studies, the need to fix cells can limit its use for non-adherent cells. Furthermore, compared to other methods, a larger number of rinsing and drying steps are required (Vichai and Kirtikara 2006). On the other hand, to the best of our knowledge, there have been no reports of interference due to the use of NPs.

Neutral Red (NR) is another reagent that functions by accumulating in the lysosomes of viable cells, generating a detectable colorimetric signal. Once crossed the extracellular membrane, the cationic NR dye interacts electrostatically with anionic or phosphate groups in the lysosomes. An adequate pH gradient is required for penetrating cell membranes and NR retention in the lysosomes, regulated mainly by ATP production. Viability is then determined based on the capacity of the cells to retain the dye, whose concentration can provide the number of viable cells (Repetto et al. 2008). This method is fast, cost-effective, and straightforward, making it widely applied in 2D models. Additionally, the NR assay is less likely to have direct chemical interactions and interferences compared to metabolic activity-based assays, such as MTT (Perez et al. 2017; Mello et al. 2020). However, its application in 3D cultures is limited by the poor penetration of the reagent into dense structures.

Within the context of cancer nanotechnology, the NR assay has been used in studies evaluating cell viability. For instance, Bakhshan et al. (2024) investigated the cytotoxic effect of hesperidin nanoemulsions on human prostate cancer cell line LNCaP. The results were consistent within the four viability methods used, demonstrating higher cytotoxicity of hesperidin-loaded nanoemulsions than free-hesperidin (Bakhshan et al. 2024). Another study employed NR to assess the cytotoxicity mediated by gold NPs coated with chitosan and containing doxorubicin (CS-AuNP-DOX). The NR assay showed that CS-AuNP-DOX presented higher cytotoxicity towards MCF-7 cells compared to free-DOX as well as a greater sensitization to radiation-combined treatment (Fathy et al. 2018).

Plasma membrane integrity-based reagents

These reagents assess cell death by detecting compromised plasma membrane integrity. The most widely used assays in this category include the lactate dehydrogenase (LDH) assay, Live/Dead assay, and exclusion-based staining methods, such as Trypan Blue and Erythrosine B.

The LDH assay measures the release of LDH from damaged cells into the extracellular medium, making it a reliable marker of cell death. It is a non-invasive method that allows repeated measurements, as only the culture medium is collected for analysis. LDH assays are suitable for both 2D and 3D models. Nevertheless, LDH assay relies on specific culture conditions, such as growth and death rate and background LDH release in control and treated samples, which can impact and alter results (Cox et al. 2021). Additionally, inconsistencies can arise in 3D systems if the medium is not collected uniformly. Variations in LDH release across different regions of the spheroid can lead to inaccurate results (Cox et al. 2021; Karassina et al. 2021). Optimizing the protocol, such as ensuring precise and consistent medium collection across samples and thoroughly mixing the medium before analysis, can mitigate these issues. Regarding NP viability assessment, it is worth mentioning that cooper-based NPs have been reported to inhibit LDH activity, leading to dose-dependent suppression of LDH detection (Kong et al. 2011).

The Live/Dead assay, commonly using reagents like Calcein acetoxymethyl ester (Calcein-AM) and Ethidium homodimer-1 (EthD-1), or fluorescein diacetate (FDA) and propidium iodide (PI) (Ross et al. 1989), is a widely employed method for assessing cell viability. The assay operates on a dual-staining principle: Calcein-AM or FDA permeates live cells, where enzymatic activity converts them into fluorescent products, emitting green fluorescence to indicate viability. In contrast, EthD-1 or PI penetrates cells with compromised membranes, binding to nucleic acids and emitting red fluorescence to mark dead cells (Dominijanni et al. 2021; Khalef et al. 2024). In 2D cultures, this method is straightforward and minimally affected by reagent diffusion, typically analyzed via fluorescence microscopy.

In 3D cultures, confocal microscopy is preferred for examining internal layers without compromising spheroid integrity. In studies conducted by Sirenko et al. (2015) and Parvathaneni et al. (2021), the Live/Dead staining assay was employed to assess cell viability in 3D models using a combination of calcein-AM (for live cells) and EthD-1 (for dead cells). In the study by Sirenko et al. (2015), the Live/Dead staining was optimized for high-throughput screening of tumor spheroids. The protocol included a one-step staining procedure to reduce variability, with an extended 3 h incubation to allow for adequate dye penetration into the spheroids. The stained spheroids were analyzed using high-content confocal imaging, with a maximum projection algorithm applied to combine multiple Z-stack images into a single 2D representation, enabling analysis of viability, apoptosis, and morphological changes. The study demonstrated differences in drug-induced cytotoxicity between 2 and 3D cultures. Similarly, in the study by Parvathaneni et al. (2021), the Live/Dead assay was used to evaluate the cytotoxic effects of transferrin-functionalized NPs in non-small cell lung cancer (NSCLC) spheroids. A comparable incubation period was applied to ensure uniform dye penetration, and imaging was performed using confocal microscopy to assess viability and structural integrity. The results confirmed the superior efficacy of the functionalized NPs in penetrating the tumor core and inducing apoptosis compared to non-functionalized formulations. Overall, Live/Dead staining combined with confocal microscopy is a reliable method for assessing cell viability in 3D models, allowing for a spatially resolved analysis of drug-induced cytotoxicity without requiring spheroid dissociation.

Exclusion-based staining methods, such as Trypan Blue and Erythrosine B, are techniques where the dye penetrates only dead cells, marking them visibly. These methods are simple and widely used for determining viability in 2D cultures. However, their application in 3D models is often limited by the poor penetration of the dye into multicellular structures (Khalef et al. 2024). Despite these limitations, exclusion-based staining methods can still be applied to 3D models if the structure is disintegrated beforehand, ensuring adequate dye accessibility to all cells. In their respective studies, Piccinini et al. (2017) and Zanoni et al. (2016) employed the Trypan Blue exclusion assay to assess cell viability in 3D cell models, but their results varied pronouncedly due to differences in methodology and the inherent limitations of the assay. Piccinini et al. (2017) conducted an evaluation of the reproducibility of Trypan Blue in both monolayers and 3D cultures, demonstrating that the method exhibited approximately 5% variability. Conversely, Zanoni et al. (2016) highlighted greater variability in Trypan Blue-based viability readings for 3D tumor spheroids, with a coefficient of variation of 42.70%, suggesting that Trypan Blue is highly prone to errors in dense 3D structures. The discrepancy between the studies might be related to differences in spheroid size, density, and dissociation efficiency before Trypan Blue staining. The limitations of this technique, which include incomplete dissociation, inconsistent staining, and potential underestimation of viability in compact structures, can impact the accuracy of cytotoxicity measurements, making it necessary to consider alternative assays better suited for such models.

Cell death assessment: autophagy, apoptosis and necrosis

One of the premises of treating cancer with nanomedicine is targeting cancerous cells and either directly killing them or offering weapons for their surroundings to do so, offering therapies with potentially fewer side effects compared to traditional approaches. However, to harness the full potential of these innovative treatments, it is crucial to understand their mechanisms of inducing cell death to evaluate not only how nanomaterials interact with cancer cells, influencing their fate and ultimately affecting treatment outcomes, but also if other non-transformed cells from surrounding tissue could also be affected by the treatment. Assays for autophagy, apoptosis, and necrosis play an important role since they are triggered by cellular processes that govern cell survival and death (D’arcy 2019). Apoptosis, often referred to as programmed cell death, is a crucial process for eliminating damaged or abnormal cells (Carneiro and El-Deiry 2020). Evaluating apoptosis aids in determining nanomedicine efficacy and understanding the underlying molecular mechanisms. Necrosis, characterized by cell swelling and rupture, typically represents an uncontrolled form of cell death associated with inflammation (Golstein and Kroemer 2007). Assessing NP-mediated necrosis can help avoid adverse reactions in surrounding healthy tissues (D’arcy 2019), enabling researchers to refine formulations for improved safety and efficacy. Autophagy, a cellular recycling mechanism, can either promote cancer cell survival or induce its demise depending on the context (Yan et al. 2019). Assessing autophagic activity helps to determine whether NPs would enhance cancer cell death or inadvertently encourage resistance.

Assessment of these mechanisms can involve a variety of methodologies. For example, structural changes to cell morphology induced by platinum NPs (produced by Penicillium pinophilum) were studied by Gholami-Shabani et al. (2023). After a 24-h treatment of HEP-G2 (hepatocellular carcinoma) cells, the authors reported chromatid condensation shrinkage and integrity of plasma membrane coherent with apoptosis; necrosis alterations could be observed by transmission electron microscopy (TEM) and surface electron microscopy (SEM) (Gholami-Shabani et al. 2023). Khan et al. (2018) designed a fluorescent magnetic submicronic polymer NP intended for breast cancer treatment and reported morphological alterations such as cell death, nuclear disintegration, and nuclear growth only after a longer exposure time (24 h), indicating a time-dependent effect (Khan et al. 2018). Both studies confirmed these results by other complementary techniques because although apoptosis, necrosis, or autophagy can be distinguished by morphologic analysis, a combination of methods is necessary to reliably confirm cell death mechanisms.

Dual apoptosis and necrosis analysis with fluorescent markers are also frequently employed to evaluate the cell death mechanisms induced by NPs. This double staining is frequently analyzed by fluorescence microscopy and/or fluorescence-activated cell sorting (FACS). A key target is phosphatidylserine, a protein located inside the plasma membrane that, during apoptosis, is translocated to the outer membrane surface, where it can be detected using fluorescently conjugated Annexin V (Miller 2004). In contrast, propidium iodide (PI) is a DNA-intercalating dye that selectively excludes viable and early apoptotic cells, serving as a marker for late-apoptotic and necrotic cells (Crowley et al. 2016). In a study conducted by Zhang et al. (2021), Annexin V and PI staining, analyzed by FACS, were used to assess the extent of apoptosis and necrosis of 4T1 murine mammary carcinoma cells after treatment with a precursor nanoemulsion (NE) and a NE containing a programmed death ligand antibody, designed to trigger an immune response after photodynamic therapy. Although the NE with the ligand antibody did not intensify cell death, treatment with 5 min-irradiation and the same NE + ligand resulted in an increased number of cells in early apoptosis (34% or 20% more) compared to non-irradiated NE + ligand-treated cells or irradiated NE with no ligand (Zhang et al. 2021).

Other stains employed to study cell death are acridine orange/ethidium bromide (AO/EB) and Hoechst. AO/EB staining is an economical alternative commonly used to differentiate early and late apoptosis and necrosis (Mironova et al. 2007). Acridine orange fluoresces green and stains live cells evenly, while early apoptotic cells exhibit bright green spots indicative of nuclear condensation and fragmentation. Ethidium bromide, which emits red fluorescence, selectively stains late apoptotic and necrotic cells; the first is visualized with the same nuclear condensation/fragmentation dots, and the latter appears more uniform with no signs of chromatin alterations. In a similar manner, Hoechst staining can also be used to observe nuclear morphological changes. However, it is usually employed as a complementary method since healthy cells are also marked by the dye and can undergo mitosis and, thus, present condensed DNA and be interpreted as false positive cell death (Crowley et al. 2016). Mohamed et al. (2024) analyzed by fluorescence microscopy the staining of AO/EB and Hoechst after treating the Caco-2 colorectal cancer cell line with quercetin-loaded solid lipid NPs (SLN) for 24 h. The group observed that treatment with SLN induced mainly late apoptosis and necrosis, as evidenced by both staining techniques (Mohamed et al. 2024).

Other DNA fragmentation detection methods can be executed to determine more accurately the damage of the treatment with a nanocarrier to the nucleus. Comet assay quantifies DNA damage through a single-cell DNA gel electrophoresis and is named after the shape left in the gel: comet’s head and damaged DNA is given by the tail—its length defines the extension of damage, since fragments are separated from intact genetic material by the gel (Collins et al. 2023). Yathindranath et al. (2022) treated U-251, 42-MG-BA, and LN-229 glioblastoma cells with lipid NPs containing a siRNA designed to target and silence the SAT1 (spermidine acetyltransferase) gene, responsible for glioblastoma resistance to irradiation. The 6 h treatment resulted in an increase in tail length by 1.5-fold compared to the control, indicating that the cells were successfully knocked down by the lipid NPs and were damaged by irradiation (Yathindranath et al. 2022).

The up- or down-regulation of several proteins can also be associated with cell death processes, and the Enzyme-Linked Immunosorbent Assay (ELISA), western blotting, flow cytometry and proteomics are some of the most common assays that evaluate its expression. Cysteine-aspartic proteases, known as caspases, are frequently evaluated as apoptotic markers and are divided into two categories. The initiator caspases (8 and 9) become active upon detecting cell damage. They trigger the activation of the executioner caspases (3, 6, and 7), initiating a cascade of events that culminate in the formation of apoptotic bodies (D’arcy 2019). To exemplify, Gokita et al. (2020) demonstrated the feasibility of delivering miRNA-634 via lipid NPs to induce apoptosis by targeting genes associated with anti-apoptosis signaling and antioxidant ability in pancreatic cancer cells. The group demonstrated that treatment enhanced the cleaved caspase-3 expression, indicating that the treatment could activate the apoptotic pathway. The High Mobility Group Box 1 protein (HMGB1), a nuclear protein released by cells that underwent unprogrammed cell death, can also be used as a marker for detecting necrosis (Gokita et al. 2020). Ding et al. (2021) analyzed HMGB1 expression in 4T1 murine breast cancer cell line to study the antitumor and antimetastatic potential of amorphous iron oxide and oxaliplatin NPs, designed as a vaccine for breast cancer, using ELISA. They reported a 3.0-fold increase of this protein in the treated group when compared to control (Ding et al. 2021).

It is worth mentioning that monitoring cell death processes based on a single morphological or biochemical assay has been oversimplified. Many phenomena associated with cell death can occur in non-cell death contexts, such as phosphatidylserine exposure occurring after trypsinization and cell scraping, processes routinely used to detach adherent cells, resulting in a false positive Annexin V signal (Nowak-Terpiłowska et al. 2021). Besides, choosing one cancer cell line as a model to study cell death processes induced by nanomaterials is not always the best option. The results can be affected by its resistance to cell death induction and heterogeneity, affecting cell death-regulatory molecules, resulting in misleading processes that would occur in vivo or in another experimental setting. One example is the deficiency of p53 by the MDA-MB-231 breast cancer cell line, an important regulator of many cell processes such as DNA repair and cell death induction, hence being a target for cancer therapy (Pozo-Guisado et al. 2002; Marvalim et al. 2023). For accurate death quantification, multiple relevant parameters must be measured (Kepp et al. 2011).

Investigation of cell signaling pathways

Understanding the mechanisms by which NPs influence cellular signaling pathways requires an approach that includes both gene and protein expressions. While protein and mRNA levels often exhibit a degree of correlation, integrating transcriptomics and proteomics offers complementary information. Gene expression begins with the transcription of DNA into messenger RNA (mRNA). Still, not all transcribed mRNA is translated into protein due to a series of regulatory factors (e.g., availability of ribosomes and amino acids). Consequently, mRNA levels only suggest possible changes in protein levels rather than demonstrating them. Therefore, it is crucial to assess both gene expression (of the specific gene of interest) and protein expression (of the corresponding protein) (Valasek and Repa 2005; Buccitelli and Selbach 2020). There are several techniques available for evaluating signaling pathways, each with its own advantages and limitations (Table 3). This section will address the main techniques for investigating gene and protein expressions.

Table 3 Main advantages and limitations of techniques used to evaluate cell signaling pathways in 2D and 3D cultures

Gene expression analysis

Gene expression involves the study of transcription levels of specific genes in cells or tissues (Buccitelli and Selbach 2020). In cancer management research, evaluating gene expression provides information on the type, level and context of gene expression. By understanding which genes are upregulated or downregulated in response to NP treatment, researchers can elucidate the mechanisms of action and therapeutic efficacy of these systems (Gavas et al. 2021). For instance, the regulation of genes involved in apoptosis, cell proliferation, and immune response can reveal whether a particular NP effectively induces cancer cell death or modulates the TME (Raju et al. 2022). Furthermore, analyzing gene expression allows for the identification of biomarkers that can predict therapeutic outcomes or resistance, facilitating the development of more targeted and personalized nanomedicine strategies (Li et al. 2021). Thus, gene expression investigation not only enhances our understanding of the interaction between NPs and tumor cells but also aids in optimizing the design of nanocarriers to improve their specificity, efficacy, and safety in cancer treatment. The investigation of gene expression can be done using techniques such as quantitative real-time polymerase chain reaction (qRT-PCR), DNA microarrays, and RNA sequencing (RNA-seq).

The qRT-PCR is the standard technique for quantifying gene expression, widely employed in cancer research. The process involves preparing cell lysates, extracting RNA, and converting it to complementary DNA (cDNA) using reverse transcriptase enzyme and oligo(dT) primers (Costa et al. 2016; Maitra Roy et al. 2023). The qRT-PCR reaction is then performed using cDNA, specific primers, and a fluorescent dye (e.g., SYBR Green, TaqMan probe), which emits fluorescence during DNA amplification. The cycle threshold (Ct) value indicates the initial quantity of the target gene, and relative expression levels are typically calculated using the ΔΔCt method, comparing the target gene to a reference gene (Valasek and Repa 2005). The study conducted by Carvalho et al. (2019) developed a colorectal ToC model, applying PCR to analyze the expression of genes related to apoptosis (Caspase-3), proliferation (Ki-67), and cellular invasion (MMP-1) following treatment with gemcitabine-loaded dendrimer NPs. They demonstrated that cells could be efficiently extracted from the chip for gene expression analysis, and revealed downregulation of these genes, indicating reduced invasion potential and proliferation, along with a delayed induction of apoptosis (Carvalho et al. 2019).

DNA microarrays enable large-scale quantitative analysis of gene expression, allowing the simultaneous measurement of thousands of genes from a single sample. The process begins with the design and fabrication of the microarray chip, followed by total RNA extraction and either labeling to form cDNA or converting into cRNA. The labeled cRNA or cDNA is then hybridized onto the chip, washed, and fluorescence is measured. The signal intensity indicates the abundance of gene transcripts, facilitating relative quantification across different conditions (Maitra Roy et al. 2023). On the other hand, RNA-seq enables identification and quantification of the transcriptome, capturing all types of transcribed RNAs, including mRNA, ncRNA, and miRNA. In the process, extraction of total RNA, fragmentation and conversion into cDNA are followed by the preparation into libraries by adding sequencing adapters and next-generation sequencing (NGS) platforms, resulting in sequence data as short or long reads. Bioinformatics analysis aligns these reads to a reference transcriptome. Differential expression analysis identifies genes with significant changes between conditions (Stark et al. 2019; Zou et al. 2022).

Building upon the advantages of RNA-seq for transcriptome analysis, a study conducted by Zou et al. (2022) demonstrated its application in evaluating the anti-tumor effects of nanoformulations in cancer research using a patient-derived organoid model from gastric cancer samples. The study compared two paclitaxel (PTX) nanoformulations: albumin-bound paclitaxel (Albu-PTX) and liposomal paclitaxel (Lipo-PTX). After treatment, total RNA was extracted, converted into cDNA, and analyzed via high-throughput sequencing on the Illumina HiSeq platform. The transcriptomic data was analyzed to identify differentially expressed genes, focusing on those involved in cancer cell survival, apoptosis, and cellular metabolism. The results indicated that treatment with Lipo-PTX significantly upregulated genes related to apoptosis and cellular stress responses (e.g., TUBA4A and TUBB2A), while downregulating genes associated with DNA replication, repair, and cell cycle progression (e.g., MCM7, BRCA2, POLE). In contrast, Albu-PTX showed a more modest effect on these pathways. Gene set enrichment analysis (GSEA) confirmed that Lipo-PTX had a stronger impact on promoting apoptosis and inhibiting DNA repair pathways compared to Albu-PTX (Zou et al. 2022).

Protein expression analysis

The investigation of protein expression extends beyond simply detecting the presence or absence of a protein and instead focuses on quantifying its abundance and localization within cells or tissues. Fundamental techniques for quantifying protein levels include western blot, which quantifies the expression of specific proteins, and mass spectrometry, enabling simultaneous quantification of thousands of proteins (Buccitelli and Selbach 2020). Moreover, immunochemistry indicates the localization of proteins within cells or tissues (Taylor et al. 2013).

Western blot

Western blot, also known as immunoblotting, is the standard technique for quantifying protein expression in cell or tissue lysates. The process involves cell lysis to release proteins, followed by reduction, denaturation and separation separated by molecular weight on a polyacrylamide gel, transferred to a membrane, and detected using specific antibodies. The binding is visualized with a secondary antibody linked to a detectable marker, and the resulting bands are quantified through densitometry to assess protein levels (Osborne and Brooks 2006; Meftahi et al. 2021). For instance, Cao et al. (2024), employed western blot to evaluate whether a cationic triblock polymer NP (PDNM) could effectively deliver siKRAS for silencing the KRAS gene in AsPC-1 pancreatic cancer cells. The results showed a significant reduction in KRAS expression, thereby confirming the efficiency of the nanocarrier in suppressing this oncogene. Additionally, there was activation of caspase-3 and caspase-9, indicating the activation of the intrinsic apoptotic pathway, while caspase-8 showed no significant change. The cleavage of PARP and reduction in Bcl-xl levels further suggested enhanced apoptosis, demonstrating that the NP system effectively induces cell death by targeting KRAS (Cao et al. 2024).

Shifting to a more complex in vitro model Huang et al. (2023) utilized MDA-MB-231 breast cancer spheroids to evaluate the anti-tumor efficacy of DNA-HCl nanogels crosslinked with disulfides, specifically using cystamine (CTM) and cystine (CYS), to enhance the delivery of doxorubicin (DOX). These modified nanogels were designed to release DOX in response to the high levels of glutathione within the TME. Western blot analysis showed that treatment with DOX-loaded DNA-HCl-CTM and DNA-HCl-CYS nanogels significantly increased levels of cleaved caspase-3, indicating the activation of the apoptotic pathway. This was accompanied by the cleavage of PARP, a hallmark of apoptosis, and a reduced expression of full-length PARP in comparison to cells treated with free DOX or non-crosslinked DNA-HCl nanogels (Huang et al. 2023). These findings indicate the importance of the western blot analysis in demonstrating that disulfide-crosslinked nanogels more effectively induce apoptosis and downregulate survival pathways in cancer cells. This, in turn, enables the development of more effective NPs for targeted drug delivery.

Immunochemistry

Immunochemistry techniques (immunocytochemistry—ICC and immunohistochemistry—IHC) employ specific antibodies to detect and visualize the presence of a protein (antigen) within individual cells (ICC) or tissue sections (IHC). The choice between ICC and IHC depends on the dimensionality of the in vitro model system being investigated. For 2D models, ICC is employed since cells are in a monolayer and can be directly assessed on microscopy slides. The biological sample is first fixed using fixatives (e.g., formaldehyde) to preserve cellular structures. Permeabilization follows, using detergents or organic solvents to allow antibodies to access intracellular antigens. Nonspecific binding is blocked with agents like bovine serum albumin. The sample is then incubated with primary antibodies specific to the target protein, followed by secondary antibodies conjugated with detectable markers to amplify the signal. Nuclear dyes, such as DAPI, are used for counterstaining. Finally, the sample is examined under a microscope with appropriate filters to detect fluorescence. The localization and intensity of staining are analyzed to determine the distribution and abundance of the target protein (Renshaw 2017). Conversely, in 3D models, IHC is preferred, requiring the spheroids/organoids to be embedded in a medium and sectioned for microscopy analysis. Two main embedding methods are used: paraffin embedding, which involves fixation, paraffin embedding, sectioning, and de-waxing, and frozen embedding, ideal for antigens sensitive to paraffin, where the model is frozen in a medium like OCT (optimal cutting temperature compound) before sectioning. Subsequently, the steps of permeabilization, blocking, and antibody incubation are performed similarly to the ICC procedure (Taylor et al. 2013; Lesavage et al. 2018; Zou et al. 2022).

To exemplify how immunochemistry can be utilized as a tool to elucidate the anti-tumor mechanisms of NPs in in vitro models, a study conducted by De et al. (2021) combined the techniques of western blotting, qRT-PCR, and ICC to elucidate the molecular mechanisms underlying cell death induced by cobalt-ferrite NPs conjugated with dopamine and functionalized with polyethylene glycol (CF-DA-PEG) in lung adenocarcinoma cells A549. These NPs were designed to deliver dopamine to cancer cells, leveraging the anti-angiogenic properties of dopamine and the magnetic properties of CF NPs for targeted delivery. Western blotting results showed increased expression of p53 and pro-apoptotic proteins Bax and cleaved caspase-3 and caspase-9 in treated cells, along with decreased levels of the anti-apoptotic protein Bcl-2. Additionally, cytochrome c release from mitochondria was detected, indicating activation of the mitochondrial apoptotic pathway. The gene expression analysis using qRT-PCR confirmed the upregulation of p53 and Bax mRNA levels, and a downregulation of Bcl-2. Finally, the ICC was used to also confirm the increased cytosolic presence of cytochrome c and p53 in CF-DA-PEG-treated cells compared to controls (De et al. 2021). Thus, by employing these three complementary techniques—western blotting, qRT-PCR, and ICC—the authors were able to conclusively confirm the activation of the mitochondrial apoptotic pathway, demonstrating the efficacy of CF-DA-PEG NPs in inducing apoptosis in cancer cells.

Mass spectrometry and proteomics

Mass spectrometry enables the identification and quantification of proteins, providing a broad view of the proteome under different experimental conditions. The process begins with cell lysis and protein extraction, followed by enzymatic digestion into peptides, typically using trypsin. In 2D models, the protein extraction process is relatively simple, often involving techniques such as detergent-based lysis and sonication. However, 3D models necessitate more elaborate sample preparation, including enzymatic treatments, to ensure efficient protein extraction. The fragmented peptides undergo separation based on their physicochemical properties. Liquid chromatography is commonly employed for this purpose, separating peptides according to characteristics like hydrophobicity or charge. After separation, the peptides are introduced into the mass spectrometer for ionization. Techniques such as electrospray ionization or matrix-assisted laser desorption ionization (MALDI) are used to convert the peptides into gas-phase ions. The mass-to-charge ratio (m/z) of peptides is analyzed to determine their identity and abundance. Finally, bioinformatics tools interpret the mass spectra, identifying peptides and inferring the presence and relative abundance of proteins (Mann et al. 2001; Aebersold and Mann 2016).

Proteomics analysis can provide a detailed molecular mechanism by which NPs could exert its anticancer effects. For instance, Buttacavoli et al. (2018) used mass spectrometry-based proteomics to elucidate the cell death pathway triggered by silver NPs embedded in exopolysaccharide (AgNPs-EPS) in SKBR3 breast cancer cells. They employed 2D-DIGE (two-dimensional difference gel electrophoresis) followed by MALDI-TOF–MS/MS (matrix-assisted laser desorption/ionization-time of flight mass spectrometry) to identify proteins with altered expression levels. The proteins were separated based on their isoelectric point and molecular weight using 2D gel electrophoresis. Protein spots that showed significant changes in expression levels were excised from the gel, digested with trypsin, and analyzed using MALDI-TOF–MS/MS. The resulting peptide fingerprints were matched against protein databases for identification. The proteomic analysis revealed significant alterations in the expression of proteins related to oxidative stress, apoptosis, mitochondrial dysfunction, and autophagy. The upregulation of proteins involved in the endoplasmic reticulum stress response and mitochondrial pathways suggested that AgNPs-EPS induces cell death via oxidative stress and mitochondrial impairment. Key proteins identified included HSP90, Bax, cytochrome C, and autophagy markers like LC3-II and beclin-1. Additionally, a reduction in glycolytic enzymes indicated a shift from the Warburg effect, potentially reducing cancer cell proliferation. The findings demonstrated that AgNPs-EPS induces cell death through oxidative stress and autophagy, with apoptosis playing a secondary role (Buttacavoli et al. 2018).

Mass cytometry

Mass cytometry, particularly cytometry by time-of-flight (CyTOF), is a technique that enables simultaneous analysis of multiple surfaces and intracellular targets at the single-cell level. CyTOF utilizes metal-labeled antibodies instead of fluorophores, allowing the simultaneous quantification of over 100 cellular parameters with minimal signal overlap, which is a common issue in traditional fluorescence-based flow cytometry (Spitzer and Nolan 2016). This method facilitates the investigation of complex signaling networks within heterogeneous cell populations, which can be particularly useful in heterogeneous tumor populations. For surface marker analysis, cells are stained with metal-tagged antibodies targeting extracellular proteins, providing information on cell identity and functional status. To assess intracellular targets, cells undergo fixation and permeabilization processes to allow antibody access to intracellular compartments (Tanner et al. 2013). This dual capability permits the concurrent evaluation of surface receptors and downstream signaling events.

A notable application of CyTOF in tumor research is demonstrated by Alföldi et al. (2019), who used the technique to analyze the expression of 12 cancer protein markers in monolayers, 3D spheroids, and in vivo tumor models of non-small cell lung cancer (NSCLC). Their results demonstrated that 3D cultures exhibited an intermediate expression pattern between monolayers and in vivo tumors. Specially, markers such as TRA-1-60, TMEM45A, pan-keratin, CD326, MCT4, Galectin-3, CEACAM5, GLUT1, and CD274 were significantly upregulated in 3D conditions compared to 2D cultures.

Building on this, Wang et al. (2021) employed CyTOF to explore the effects of chemotherapy on breast cancer stem cells (BCSCs) and the potential of nanotechnology to enhance treatment efficacy. Using MDA-MB-231 and 4T1 breast cancer spheroids, the researchers modeled BCSC enrichment to mimic tumor heterogeneity and drug resistance. The CyTOF analysis involved staining cells with a panel of metal-conjugated antibodies targeting stemness, epithelial-mesenchymal transition, and drug resistance markers. After doxorubicin treatment, CyTOF identified a significant upregulation of stemness markers (CD90, CD133, SOX9, Nanog) and epithelial-mesenchymal transition-associated proteins (vimentin), indicating that chemotherapy induced stem-like properties in BCSCs. However, when doxorubicin was combined with chitosan-modified (CS-V) NPs, this effect was reversed, suppressing stemness and enhancing cytotoxicity. The advanced computational tools t-SNE and SPADE clustering confirmed the reduction in stem-like subpopulations upon CS-V NPs treatment, demonstrating that CyTOF is a powerful tool for profiling drug-induced phenotypic changes at a single-cell level.

Recent advancements include imaging mass cytometry (IMC), which combines CyTOF technology with laser ablation to achieve spatially resolved protein expression data within tissue/3D structure sections. IMC enables visualization of protein distributions at subcellular resolution, providing information on tissue architecture and microenvironmental influences on cell signaling. Bouzekri et al. (2019) employed IMC to analyze drug-treated breast cancer cells, including SK-BR-3, HCC-1143, and MCF-7 cell lines, to assess phenotypic and molecular responses to three compounds: epidermal growth factor (EGF), nocodazole, and etoposide. The IMC methodology involved staining cells with metal-labeled antibodies targeting surface, cytoplasmic, and nuclear markers, followed by laser ablation and ion detection via time-of-flight mass spectrometry. Computational image analysis enabled the extraction of single-cell quantitative data and spatial heatmaps. The results demonstrated that IMC effectively distinguished phenotypic heterogeneity among cell lines and drug-induced responses, revealing EGF-induced mesenchymal transition, nocodazole-mediated mitotic arrest, and etoposide-induced DNA damage and apoptosis. High-dimensional clustering and hierarchical analysis identified drug-specific biomarker correlations, highlighting IMC’s ability to detect subtle cellular changes with high multiplexing capacity.

Oxidative stress evaluation

Reactive oxygen species (ROS) and reactive nitrogen species (RNS) are byproducts of cellular metabolism (e.g., mitochondrial metabolism and phagocytosis). However, their excessive or dysregulated production can lead to a cellular state known as oxidative stress. Oxidative stress is a condition characterized by an imbalance between the production of ROS/RNS and the ability of cells to neutralize them with antioxidant defense systems. This imbalance leads to the accumulation of ROS (e.g., superoxide anion radical [O₂·^_], hydrogen peroxide [H₂O₂], hydroxyl radicals [HO·]), and RNS (e.g., nitric oxide [NO·], peroxynitrite anion [ONOO^_]), which possess destructive potential, leading to oxidative damages to macromolecules including proteins, lipids, and nucleic acids, causing cellular dysfunction (Halliwell et al. 1992; Birben et al. 2012; Sies 2015).

In the context of cancer, ROS play a dual role depending on their concentration, exhibiting anti-tumor or pro-tumor actions. They can damage DNA within cancer cells, leading to genetic mutations that drive cancer progression. ROS can also stimulate angiogenesis within TME. This newly formed vasculature facilitates the delivery of oxygen and nutrients, which are essential for tumor sustenance (Sosa et al. 2013; Hayes et al. 2020). However, when ROS are intensively generated within the TME and overwhelm the antioxidant defense system, a switch towards anti-tumor effects can occur. In this scenario, the elevated levels of reactive species can extensively damage macromolecules, inducing a state of intense oxidative stress. Furthermore, ROS can disrupt cell cycle checkpoints, leading to cell cycle arrest (Sosa et al. 2013). ROS can activate immune cells, promoting anti-tumor immune surveillance, ultimately leading to apoptosis or ferroptosis of cancer cells (Redza-Dutordoir and Averill-Bates 2016; Dodson et al. 2019).

Monitoring the levels of oxidative stress within TME can serve as an indicator of the efficacy of nanocarrier-based therapies. This section will outline methods for evaluating oxidative stress in both 2D and 3D tumor culture models. These methods include: (i) direct measurement of reactive species (ROS and RNS); (ii) assessment of antioxidant defense system; (iii) quantification of oxidative damage to proteins, lipids, and DNA.

Direct detection of reactive species

Accurate ROS/RNS detection requires measurements in intact cells, as analyzing disrupted samples like tissue homogenates can alter their levels due to the short lifespans of these reactive species (Murphy et al. 2022). Fluorescent probes are commonly used to detect ROS/RNS via spectrofluorometry, microscopy, and flow cytometry. For non-specific detection of ROS, DCFH-DA (2’,7’-dichlorodihydrofluorescein diacetate) is frequently employed, converting to fluorescent DCF upon oxidation, though its sensitivity can be affected by factors like oxygen levels and pH (Kalyanaraman et al. 2012). Superoxide-specific detection often relies on dihydroethidium (HE) or its mitochondrial-targeted version MitoSOX, but distinguishing specific fluorescence requires liquid chromatography-mass spectrometry (LC–MS) due to overlapping emissions from non-specific oxidation products (Zielonka et al. 2008; Shchepinova et al. 2017).

For specific detection of hydrogen peroxide, phenylboronate- and borinic acid-based probes are commonly used, with borinic acid offering better sensitivity. However, both can react with other species, potentially leading to inaccuracies. Validation using nitric oxide synthase (NOS) inhibitors or catalase can help clarify results. Genetically encoded probes like HyPer and roGFP2 offer higher sensitivity and specificity for H₂O₂ detection in live cells (Zielonka et al. 2008; Kalyanaraman et al. 2012; Shchepinova et al. 2017; Winterbourn 2018; Murphy et al. 2022). Specific detection of ONOO^_ presents another challenge. While phenylboronate-based probes, similar to those used for H₂O₂ detection, can be employed along with the probe of the enzyme catalase to eliminate confounding H₂O₂ signal, this method is not without limitations. Catalase treatment may not completely remove all H₂O₂, potentially leading to an underestimation of ONOO^_. Dihydrorhodamine (DHR) is another option, but it can react with multiple reactive species (Kalyanaraman et al. 2012). Further, nitric oxide (NO·) detection typically uses diaminofluorescein (DAF-2) probes, which produce fluorescent triazole derivatives (Hardy et al. 2018).

Exemplifying the use of ROS/RNS detection techniques to evaluate the anti-tumor effects of NPs, Liu et al. (2020b) utilized ROS direct detection to investigate the anti-tumor effect of PLGA-PEG NPs loaded with cinnamaldehyde (CA) and diallyl trisulfide (DATS) in MDA-MB-231 human breast cancer cell line. Using DCFH-DA as a probe, they observed significantly elevated ROS levels with CA-DATS-loaded NPs compared to single-agent treatments. The ROS increase was further confirmed by a reduction in the mitochondrial membrane potential, indicating oxidative stress-induced mitochondrial damage. The elevated ROS levels led to the activation of apoptotic pathways, as evidenced by increased staining for cleaved caspase-3 and the release of cytochrome c into the cytosol. The combined use of CA and DATS showed a synergistic effect, significantly amplifying ROS production and leading to increased cytotoxicity in MDA-MB-231 cells (Liu et al. 2020b).

Detection of antioxidant defense system levels

To safeguard against the accumulation of ROS and RNS, cells contain a sophisticated arsenal of antioxidants, including both non-enzymatic and enzymatic compounds. They work in concert to maintain cellular redox homeostasis (Halliwell et al. 1992). Endogenous antioxidants include bilirubin, α-lipoic acid, melatonin, melanin, uric acid, and glutathione (GSH). Among them, GSH acts as both a direct scavenger and as a cofactor for enzymatic antioxidants. In TME, GSH levels are often elevated due to the heightened oxidative stress associated with cancer cell proliferation (Halliwell et al. 1992; Gamcsik et al. 2012). Additionally, antioxidant enzymes, including superoxide dismutases (SODs), catalase (CAT), and glutathione peroxidases (GPXs), act sequentially to neutralize various reactive species (Birben et al. 2012; Gill et al. 2016; Dodson et al. 2019).

GSH levels are commonly measured using colorimetric and fluorometric assays. The DTNB assay, a popular colorimetric method, detects GSH by its reaction with 5,5’-dithiobis-2-nitrobenzoic acid, although it can be affected by other thiol-containing compounds (Demirci-Çekiç et al. 2022). In addition, enzymatic recycling methods offer a more specific approach to GSH determination. These techniques use enzymes like GR and glutathione S-transferase (GST) to convert GSH into a detectable product, involving the reduction of GSSG to GSH by GR, followed by the reaction of the regenerated GSH with DTNB to form the DNTB-GSH adduct (Rahman et al. 2007). Alternatively, fluorometric assays, such as the o-phthalaldehyde (OPT) assay, detect GSH by forming fluorescent adducts (Senft et al. 2000). Furthermore, LC–MS is considered the most accurate technique for distinguishing GSH from other thiol species (Yu et al. 2015).

Regarding antioxidant enzymes, the assessment of mRNA or protein expression levels may not correlate with their activity, and evaluating enzyme activity directly provides a more accurate assessment of the functional status of the antioxidant defense system (Weydert and Cullen 2010). Enzymatic assays involve monitoring the conversion of a substrate into a product over a specified period. The rate of product formation correlates with enzyme activity, allowing for quantitative assessment. Various methods can be employed. For instance, SOD activity can be measured using the xanthine-xanthine oxidase/nitroblue tetrazolium (NBT) assay (Spitz and Oberley 1989, 2001), while zymography qualitatively visualizes enzyme activity within a gel matrix (Weydert and Cullen 2010).

In cancer research, Hu et al. (2023) demonstrated the role of GSH depletion in enhancing NP-induced oxidative stress. They treated MCF-7 and MCF-7/ADR breast cancer cells with α-tocopherol succinate dimer NPs combined with doxorubicin, showing that reduced intracellular GSH levels led to increased ROS, mitochondrial dysfunction, and apoptosis, ultimately enhancing doxorubicin cytotoxicity, particularly in drug-resistant cells (Hu et al. 2023). Similarly, Doktorovová et al. (2014) explored the impact of cationic solid lipid NPs (cSLNs) on antioxidant enzyme activities in HepG2 hepatocellular carcinoma cells. The treatment increased SOD and GPx activities as a cellular defense mechanism against ROS but reduced GR activity, impairing GSH regeneration and weakening antioxidant defenses over time. This imbalance heightened cancer cell susceptibility to oxidative damage (Doktorovová et al. 2014).

Detection of oxidative stress biomarkers

Cellular exposure to ROS and RNS can cause oxidative damage to macromolecules, including proteins, lipids, and DNA, with the resulting subproducts serving as biomarkers for assessing the extent of oxidative stress (Demirci-Çekiç et al. 2022). One hallmark of protein damage is carbonylation, marked by the introduction of carbonyl groups into amino acids. The widely used protein carbonyl assay involves derivatizing these groups with 2,4-dinitrophenylhydrazine (DNPH) for spectrophotometric quantification. However, for greater specificity and sensitivity, liquid chromatography-mass spectrometry (LC–MS) is preferred, as it can detect multiple oxidative modifications simultaneously. Complementary techniques such as ICC/IHC and western blotting can further localize and quantify oxidized proteins within cells (Rabbani and Thornalley 2020; Murphy et al. 2022).

Lipid peroxidation, another consequence of ROS exposure, disrupts membrane integrity and can trigger inflammation, apoptosis, and ferroptosis (Dodson et al. 2019). Common biomarkers of this process include 4-hydroxynonenal (4-HNE) and malondialdehyde (MDA) (Waeg et al. 1996; Jaganjac and Zarkovic 2022). 4-HNE can be identified through ICC/IHC using specific antibodies (Waeg et al. 1996), while MDA is typically detected using the thiobarbituric acid-reactive substances (TBARS) assay (Esterbauer et al. 1991). Although the TBARS assay is convenient, it lacks specificity, making LC–MS a more accurate method for lipid peroxidation measurement (Li et al. 2019). Additionally, oxidative DNA damage occurs through base degradation, strand breaks, and cross-linking, which can lead to mutations and cell death. Techniques such as the comet assay, which detects DNA strand breaks by visualizing “comet-like” structures under microscopy (Muruzabal et al. 2021), and agarose gel electrophoresis, which separates DNA fragments by size, are commonly used (Lee et al. 2012). The oxidative biomarker 8-hydroxy-2’-deoxyguanosine (8-OHdG) is often measured using ultra-performance LC–MS/MS, as enzyme-linked immunosorbent assays (ELISAs) may produce inconsistent results (Birben et al. 2012; Jaganjac and Zarkovic 2022). For 3D models, IHC also can be used for identifying cells with elevated 8-OHdG levels (Toyokuni et al. 1997).

In NP research, oxidative stress biomarkers are critical for understanding NP-induced cytotoxicity. Alili et al. (2013) investigated protein oxidation mediated by dextran-coated cerium oxide NPs (CNPs) in A375 melanoma cells, measuring protein carbonylation using DNPH derivatization and western blotting. They found a significant increase in protein carbonylation, indicating elevated oxidative stress, which was more pronounced in melanoma cells than in normal stromal cells, indicating the selective cytotoxicity of CNPs. This oxidative damage was linked to increased ROS levels, mitochondrial dysfunction, and activation of apoptotic pathways, ultimately reducing melanoma cell viability (Alili et al. 2013). In another study, Alarifi et al. (2014) studied the effects of iron oxide NPs (IONPs) on oxidative stress biomarkers in MCF-7 breast cancer cells, assessing lipid peroxidation through the MDA assay and DNA damage via the comet assay. They found that IONPs significantly increased MDA levels and induced DNA strand breaks in a dose- and time-dependent manner, indicating increased oxidative stress. The study concluded that IONPs cause oxidative damage to macromolecules, contributing to cancer cell cytotoxicity (Alarifi et al. 2014).

Assessment of cell migration and invasion

One of the biggest concerns in cancer is metastasis, a complex process that relies on the interactions between cell–cell and cell–matrix, tumor and tissue microenvironment, and intrinsic tumor cell properties, including its capacity to migrate and invade adjacent tissues (Welch and Hurst 2019). Cellular migration can happen by single or collective migration and is often described as the movement of cells across the body, while invasion is the capability of the cell to penetrate neighboring tissue while modifying its environment (Decaestecker et al. 2007; Apolinário et al. 2020; Novikov et al. 2021). To study whether treatments are hindering or altering the tropism of cells, there are several models available, the most common 2D and 3D models will be discussed in the following subsections. Table 4 provides an overview of the main methods to study cell migration, outlining their respective costs, complexity, and limitations.

Table 4 Main advantages and limitations of techniques commonly used to evaluate cell migration and invasion

Scratch/wound-healing assay

In this assay, a gap is created using a scraping tool or a micropipette tip in confluent monolayered cells; this gap can be measured after treatment by assessing the distance between the edges of the gap, the cell-free area dimensions or by cell counting within the scratch. This methodology can be used as an inexpensive and easy alternative to study whether the nanocarrier can alter cellular events leading to wound closure, although it does not distinguish gap closure due to cell migration and proliferation. To specifically assess migration, the addition of antimitotic agents, such as mitomycin C, is commonly employed. These agents inhibit cell proliferation without affecting migratory capacity, allowing researchers to differentiate between the effects of migration and proliferation in wound closure (Martinotti and Ranzato 2020). In the context of cancer research, for example, Liu et al. (2021a) studied the influence of targeted cerium oxide NPs on breast cancer metastasis, and to evaluate the migration of 4T1 cells, the authors used the scratch assay. Compared to control, the formulation was found to reduce cell migration in a dose-dependent manner, reaching up to 80% reduction with a 200 nM of the targeted NPs treatment (Liu et al. 2021a).

Additionally, although Daré et al. (2024) did not focus on antitumor effects, they employed the assay to evaluate the potential of nanostructured lipid carriers co-encapsulating simvastatin and adenosine (NLCs-S/A) in promoting wound healing. In a setup without mitomycin C, NLCs-S/A achieved 85.9% wound closure, significantly outperforming the control. In contrast, when mitomycin C was included to isolate migration effects, NLCs-S/A achieved 39.3% wound closure (Daré et al. 2024). This approach demonstrates the importance of incorporating antimitotic agents when using the scratch assay to assess cell migration. Without this distinction, the assay may overestimate the contribution of migration, particularly when treatments also influence cell proliferation. Despite its limitations, the scratch assay remains an initial screening tool for evaluating the effects of nanocarriers on cellular migration. Its simplicity and adaptability make it particularly useful for high-throughput screening, especially when combined with imaging technologies and quantitative analysis tools that enhance accuracy and reproducibility (Yarrow et al. 2004).

Transwell migration/invasion assay

The transwell assay is a widely used technique for studying cell migration and invasion. This assay involves a permeable membrane with defined pore sizes placed between two chambers. Cells are seeded in the upper chamber, and the lower chamber contains a medium that may include chemo-attractants, such as fetal bovine serum (FBS), creating a concentration gradient to stimulate directed cell movement. For migration assays, the membrane is uncoated, and cells that traverse the membrane are stained usually with DAPI, Hoechst, crystal violet or hematoxylin, and counted either manually or using software (Pijuan et al. 2019).

For invasion assays, the insert membrane is coated with Matrigel or another collagen-based hydrogel to simulate the ECM, enabling the evaluation of invasive capacity of cells. Cells can be monitored using an inverted microscope or by fixing the matrix and cutting it on microtome. This method helps identify the depth profile of the cell and the influence of the treatment on invasion, and can be considered a 2.5D assay, since there is penetration on the gel and cells can migrate in groups (Kramer et al. 2013). To the best of our knowledge, we are unaware, to date, of studies that employed this model to study the effects of nanocarriers.

The transwell assay is a cost-effective and versatile method for evaluating migratory and invasive behaviors of various cell types. However, it primarily assesses individual cell migration or invasion, making it less informative about collective cell movement. Variability in membrane coating and cell scraping can affect consistency, and manual cell quantification may introduce observer bias. For invasion analysis, it is essential to confirm that cells can invade the membrane and Matrigel coating, as some cell types may migrate horizontally but lack the ability to penetrate porous membranes (Trepat et al. 2012; Pijuan et al. 2019). An example of NP evaluation using the transwell assay to assess migration is provided by Salata et al. (2021), who demonstrated that a phosphatidylcholine-based microemulsion containing fenretinide intended for breast cancer chemoprevention reduced cell migration by 75.9% in the transwell model, indicating that the formulation exerted a migrastatic effect on T-47D cells, supporting its chemopreventive role (Salata et al. 2021).

Microfluidic chips

Microfluidic chips represent a powerful platform for studying cell migration and invasion under precisely controlled conditions. These devices come in various configurations, typically involving a network of microchannels connected to an electronic system to control optimal parameters such as fluid flow, pressure, and gradient formation. Cells are plated in designated wells or inlets and are expected to migrate through the channel either toward the outlet or through the microchannel itself once a wound is made with trypsin in one channel (Nie et al. 2007; Dou et al. 2022). One of the main advantages of microfluidic chips is their ability to provide precise control over the experimental environment. Additionally, real-time monitoring of cell behavior at the single-cell level can be achieved, allowing researchers to capture migration dynamics, directionality, and interactions between cells and their surroundings (Nie et al. 2020). Sheykhzadeh et al. (2020) assessed the influence of transferrin-conjugated porous silicon NPs on the migration of U87 glioblastoma multiforme cells by microfluidic-based migration chip. Their study revealed that NP uptake was associated with a 40% reduction in cell migration, highlighting the potential of NPs to inhibit cancer cell dissemination (Sheykhzadeh et al. 2020).

Despite their advantages, microfluidic chips face several limitations that must be addressed to maximize their potential. Achieving optimal conditions for experiments, such as maintaining appropriate flow rates, chemical gradients, and pressure, often requires extensive optimization and technical expertise. Another limitation involves difficulties in working with low-dose treatments, as small variations in concentration may lead to inconsistent results due to the microscale nature of the system (Nie et al. 2007).

Cell tracking

Cell tracking consists of recording living cell movements over time, and the trajectory can be tracked manually or automatically via software. In 3D models, cells are embedded in ECM-like environments, such as Matrigel or collagen gels, and are tracked in all spatial dimensions (X, Y, and Z coordinates) to provide a view of their migratory behavior (Decaestecker et al. 2007). This method is particularly valuable in studying collective or single-cell migration, revealing how cells respond to external stimuli, such as NPs, chemokines, or growth factors. However, some limitations to this method can be mentioned. First, live imaging systems are required, which may be costly and require expertise in handling. Additionally, software for accurate tracking can be complex and may require proper calibration for different experimental conditions. Imaging artifacts, such as photobleaching, background noise, or cell overcrowding, may impair tracking accuracy, particularly in dense or highly dynamic cell populations. For manual tracking, high cell density and long observation times may make the process laborious and prone to errors. Moreover, limitations in resolution can make distinguishing between closely packed cells difficult, especially in dense 3D cultures (Menyhárt et al. 2016).

To overcome these challenges, several optimization strategies have been proposed. Automated tracking algorithms can be fine-tuned using machine learning approaches to improve cell segmentation and minimize errors due to overlapping or faintly stained cells. Imaging conditions, such as light intensity and frame rates, can also be optimized to reduce photobleaching and capture continuous movements without missing key events. For 3D models, confocal or multiphoton microscopy may offer better spatial resolution and depth penetration compared to traditional widefield microscopy (Kok et al. 2020). Furthermore, combining cell tracking with additional functional assays, such as viability or gene expression analysis, provides a more comprehensive understanding of how NPs or treatments affect cellular migration.

Clonogenic assay—assessment of cell survival

The clonogenic (or colony formation) assay is an in vitro cell survival assay that evaluates the ability of cells to grow into a colony (i.e., 50 or more cells) after exposure to ionizing radiation, chemical compounds (such as cytotoxic agents) or genetic manipulation (Franken et al. 2006; Rafehi et al. 2011). It differs from other cell survival and proliferation assays, as it allows the assessment to be carried out over extended periods (Hillegass et al. 2010). Thus, the clonogenic assay directly measures the number of viable cells rather than the synthesis of protein, DNA, RNA, or mitochondrial enzymes evaluated in other cytotoxicity assays, which are cell components that can be impacted without any change in cell number (Herzog et al. 2007). In the context of cancer therapy, clonogenic assays are the gold standard method to evaluate cancer cells’ response to radiation (Matsui et al. 2019). Furthermore, it is shown to be an advantageous method for evaluating the effect of NPs on cell viability and proliferation (i.e., colony number and size) as it does not require fluorescent or colorimetric indicators that could be impacted by nanomaterial interference (Hillegass et al. 2010).

To date, the most common form of clonogenic assays to assess NPs toxicity is performed on cell monolayers. However, a few recent publications reported investigations of colony formation on 3D cell cultures (spheroids) and on-a-chip models (Lee et al. 2017; Brüningk et al. 2020; Chermat et al. 2022). The most used clonogenic assay protocol in cell monolayers is based on seeding the cells at clonal density before or after NPs treatment. After 1–3 weeks, colonies are fixed, stained, and counted to evaluate the fraction of seed cells that maintain the ability to form colonies (Franken et al. 2006; Hillegass et al. 2010; Rafehi et al. 2011). Herzog et al. (2007) evaluated the toxicity of three types of carbon-based NPs by the clonogenic assay on carcinoma and normal bronchial epithelial cell lines (A549 and BEAS-2B, respectively). As mentioned before, by employing the clonogenic assay, carbon interactions with colorimetric indicators could be avoided; the study endpoints were colony number and surface area. After 10 days, all three types of carbon NPs significantly decreased clonogenic survival and proliferation in both cell lines. Notably, the effects were more pronounced in normal cells compared to A549 lung carcinoma cells (Herzog et al. 2007). The clonogenic assay was also used to evaluate the cytotoxic effects of titania and alumina NPs in A549 cells (Wei et al. 2014). The authors observed a reduction in cell number within colonies that was significantly dependent on time and NPs size; smaller particles (5–10 nm) reduced more colony formation than larger particles (50–200 nm).

Seeking to understand how the cellular model influences the formation of colonies after treatment with NPs and radiation, Popescu et al. (2023) used the clonogenic assay to evaluate the dual chemotherapy-radiosensitization efficiency of doxorubicin-loaded iron oxide NPs in both monolayer and spheroids models of human cervical adenocarcinoma (HeLa) and human squamous cell carcinoma (FaDu). For cell monolayer survival evaluation, cells were seeded and treated with NPs for 16 h prior to radiation therapy (150 kV X-rays). In turn, spheroids were obtained using the liquid-overlay technique, following incubation with NPs for 48 h and irradiation with X-rays; NP treatment was longer for spheroids to allow greater penetration effectiveness towards the center of the spheroid. After radiation treatment, the cells were detached and seeded at clonal density (200–500 cells/well) in 6-well plates for 14 days. The response of tumor cells to NP and radiation therapy was highly dependent on the cell model. Spheroids of FaDu cells presented a tighter morphology, resulting in reduced NPs penetration. Thus, although treatment with NPs reduced colony formation at all concentrations studied in 2D models, only the highest concentration induced cytotoxic effects in FaDu spheroids. HeLa spheroids, in turn, presented a looser morphology allowing a greater degree of NPs penetration through the whole spheroid structure. Thus, the response of tumor cells to NPs treatment followed by ionizing radiotherapy was similar in 3D and 2D models (Popescu et al. 2023).

Models combining cell spheroids and on-a-chip platforms for clonogenic evaluation have been recently developed, although studies assessing the effects of NPs on colony formation in these models are still missing. In the model developed by Lee et al. (2017), a microwell chip was used to grow spheroids immobilized in alginate on top of micropillars. The cells were immersed in microwells containing growth medium and treatments prior to staining with a dye solution and scanning of the micropillar chip for colony analysis (Lee et al. 2017). Another described methodology is to collect spheroids from the microfluidic device after treatment, dissociate them into single cells, and seed them in 12-well plates. After 10–14 days of incubation, colonies can be fixed, stained, and counted (Patra et al. 2019; Brix et al. 2021). These models aim to overcome technical challenges in evaluating colony formation in 3D assays, in which the presence of a thick ECM can hinder image quality and colony counts. However, the applicability of this model still needs to be investigated in the context of nanostructured systems.

Finally, low in vitro clonal growth rates may indicate a good prognostic factor and disease-free survival (Aapro 1985; Brix et al. 2021). With the emergence of NPs for cancer treatment, clonogenic assays that produce reproducible, comparable, and representative results of in vivo toxicity are valuable to select treatments that induce a higher remission rate. Table 5 summarizes existing assays used to assess the ability of NPs to alter colony formation using in vitro models and the main advantages and limitations of these techniques.

Table 5 Overview of the main in vitro clonogenic assay models

Cytoskeleton evaluation

Cytoskeletal proteins, including microtubules (α- and β-tubulin heterodimers), microfilaments (actin), and intermediate filaments (e.g., vimentin, keratins), are essential for cell proliferation and physiology. In the context of tumors, cytoskeletal proteins can be studied to analyze the effects of antitumor agents (Ong et al. 2020; Zhang et al. 2022). Microfilaments of actin, existing in monomeric (G-actin) and polymeric (F-actin) forms, regulate cell motility, adhesion, and morphology and are involved in cell motility and invasion, processes often dysregulated in cancer cells (Ong et al. 2020). Microtubules are involved in cell division, intracellular transport, and maintaining cell shape and are targeted by antitumor agents to disrupt microtubule dynamics to inhibit tumor proliferation (Ong et al. 2020). Intermediate filaments, including vimentin, cytokeratin, and desmin, provide mechanical stability and are cell-type specific. In cancer, intermediate filament reorganization can induce epithelial-mesenchymal transition, promoting invasion and metastasis (Satelli and Li 2011; Ong et al. 2020). Various types of NPs (such as carbon-based, gold, and polystyrene NPs) have the ability to directly interact with cytoskeletal components or carry molecules that induce cytoskeletal alterations, resulting in cell cycle arrest and apoptosis (Ispanixtlahuatl-Meráz et al. 2018; Xiao et al. 2019; Zhang et al. 2022). In vitro cytoskeleton assays can provide information on how such nanomedicines may change cytoskeleton organization, contributing to the development of new strategies for cancer therapy (Table 6).

Table 6 Main advantages and limitations of techniques used to evaluate cytoskeleton in 2D and 3D cultures

The assessment of the cytoskeleton in vitro in monolayers and 3D models can be performed using fluorescence microscopy-based imaging techniques, biochemical methods (such as RT-qPCR and Western blot for the quantification of cytoskeletal gene and protein expression, see Sect. “Investigation of cell signaling pathways” for further details), flow cytometry to analyze cytoskeletal-related markers at the single-cell level, and biophysical approaches for analyzing mechanical properties and structural dynamics.

Fluorescence microscopy-based methods are widely used for visualizing cytoskeletal structures, with immunofluorescence staining targeting actin, tubulin, or vimentin being a standard approach. Confocal microscopy is particularly advantageous for cytoskeletal studies due to its ability to eliminate out-of-focus light, providing high-resolution images with high sensitivity and specificity for cytoskeletal protein visualization (McKayed and Simpson 2013). However, a notable limitation of this technique is prolonged phototoxicity and photobleaching, especially during long-term imaging, which can compromise cell viability and image quality (Laissue et al. 2017). To overcome these limitations in dense 3D models, light sheet microscopy is a superior alternative. This technique illuminates the sample with a thin sheet of light, capturing an entire plane simultaneously, thereby minimizing photobleaching and phototoxicity while enabling rapid, high-resolution imaging of cytoskeletal components such as actin filaments, microtubules, and intermediate filaments (Reynaud et al. 2008). Despite these advantages, light sheet microscopy can present technical challenges, including meticulous optical alignment and the requeirement of specialized equipment, making accessibility a challenge (Stelzer et al. 2021; Olarte et al. 2018). In addition to being employed for cytoskeletal studies, fluorescence microscopy-based methods are also employed to track NP uptake, assess subcellular localization and intracellular trafficking in live-cell models.

Beyond fixed-cell imaging, live-cell imaging techniques enable real-time visualization of cytoskeletal dynamics. Fluorescent cytoskeletal reporters, such as LifeAct-GFP for F-actin and Tubulin-GFP for microtubules, allow real-time visualization of cytoskeletal dynamics. These genetically encoded probes bind to specific cytoskeletal components without disrupting their function, making them ideal for monitoring actin polymerization, stress fiber formation, microtubule polymerization, and spindle dynamics during cell division, among other processes (Riedl et al. 2008; Yamamoto et al. 2001).

To exemplify, Lei et al. (2015) utilized this technique to assess cytoskeletal alterations induced by paclitaxel-loaded expansile NPs (Pax-eNPs) in MSTO-211H malignant mesothelioma spheroids. These NPs were designed to enhance drug delivery through pH-responsive swelling, improving intratumoral retention. Cytoskeletal analysis, performed via confocal fluorescence microscopy with actin staining, revealed that Pax-eNPs effectively penetrated tumor spheroids and exhibited superior drug retention compared to conventional paclitaxel formulations. Over time, significant cytoskeletal disruptions emerged, coinciding with increased apoptosis and necrosis, suggesting that cytoskeletal destabilization was involved in the prolonged cytotoxic effects of Pax-eNPs.

A different approach was taken by Lorenzo et al. (2011), who employed Selective Plane Illumination Microscopy (SPIM), a light-sheet microscopy technique, in HCT116 colon carcinoma spheroids expressing histone H2B fused to HcRed, allowing for high-resolution imaging of mitotic dynamics. SPIM enabled the real-time visualization of chromosome condensation and spindle formation during mitosis with minimal phototoxicity. The study demonstrated that mitotic cells could be identified within spheroids, showing distinct cytoskeletal changes such as chromosome alignment and cleavage plane orientation. Additionally, the effect of paclitaxel treatment was assessed, revealing mitotic arrest and prolonged chromosome condensation, confirming its impact on microtubule dynamics. The study showed that SPIM provides advantages over confocal microscopy, offering deeper tissue penetration, reduced photobleaching, and high spatial resolution.

Flow cytometry can be used to analyze cytoskeletal proteins and structural rearrangements by measuring changes in cellular properties that indirectly reflect cytoskeletal dynamics. This approach is predominantly applied to monolayers. Shin et al. (2021) employed a NP uptake assay to investigate cytoskeletal rearrangement in HaCaT keratinocyte cells. Their methodology was based on the premise that cytoskeletal reorganization influences membrane dynamics and endocytic activity, thereby affecting the internalization of NPs. To induce cytoskeletal modifications associated with epithelial-to-mesenchymal transition (EMT), cells were treated with TGF-β1, a well-known EMT inducer that disrupts cell–cell adhesion and promotes actin cytoskeletal remodeling. After treatment, cells were exposed to fluorescently labeled silica NPs (FITC-SiO₂NPs), which were taken up by the cells in a manner dependent on cytoskeletal organization. Following incubation, cells were subjected to flow cytometry, where FITC fluorescence intensity was quantified to assess NP internalization. Adose-dependent increase in NP uptake in TGF-β1-treated cells was observed, correlating with cytoskeletal remodeling. This observation suggests that cytoskeletal rearrangement alters the cell surface properties and endocytic pathways, leading to enhanced NP internalization. Thus, by measuring the extent of NP uptake, flow cytometry provided an indirect but quantitative assessment of cytoskeletal alterations. Flow cytometry may be applied to 3D cultures with adaptations, such as dissociating cells into single-cell suspensions, altough this process may introduce artifacts and compromise the preservation of the 3D architecture.

Biophysical techniques, such as Atomic Force Microscopy (AFM), can also be employed to measure cytoskeleton-related mechanical properties, including cell stiffness, contractility, and elasticity at the nanoscale. AFM provides high-resolution, quantitative data on the mechanical behavior of cells, offering information about cytoskeletal dynamics and their role in cellular function. However, the interpretation of AFM data can be complex and often requires specialized expertise to accurately correlate mechanical measurements with cytoskeletal organization and activity (Jalili and Laxminarayana 2004). To exemplify its use in monolayers, Xiao H. et al. (2019) employed AFM to assess cytoskeletal alterations induced by AgNPs in colon cancer cells (HCT-116). Before exposure, fluorescence imaging revealed well-organized cytoskeletal structures, while AFM images showed smooth membrane surfaces with clearly defined lamellipodia. After AgNP treatment, AFM images exhibited cytoskeletal degradation, disorganized lamellipodia, and increased membrane roughness, suggesting severe structural damage. Quantitative analysis confirmed a dose-depenedent membrane roughness, suggesting a direct correlation between AgNP exposure and cytoskeletal alteration, while. cell adhesion and stiffness were significantly reduced, which correlated with decreased cell proliferation and viability. Similarly, Guryanov et al. (2020) analyzed the cytoskeletal organization and biomechanical properties of Caco-2,,HCT-116 non-malignant mesenchymal stem cells (MSC) and human skin fibroblasts (HSF) after treatment with prodigiosin-loaded halloysite nanotubes (p-HNTs). AFM imaging revealed that p-HNTs strongly adhered to cancer cell membranes, causing F-actin disorganization, while non-malignant cells maintained intact cytoskeletal structures. These findings suggest that p-HNTs selectively disrupt the cytoskeleton of malignant cells, impairing their structural integrity and promoting cell detachment.

AFM has also been successfully applied to 3D models. Andolfi et al. (2019) investigated the biomechanical properties and cytoskeletal organization of two distinct biological systems: tumor spheroids derived from human non-small cell lung carcinoma (NSCLC, A549 cell line) and human oocytes. The tumor spheroids (130–325 µm in diameter) exhibited a bimodal relaxation behavior, where the outer proliferative layer influenced the viscoelastic properties while the inner core remained structurally stable. In contrast, oocytes at different maturation stages (MI and MII) exhibited distinct mechanical profiles.

Key aspects of methodological approaches

The methodologies used to evaluate NPs uptake and effects in cancer research are very diverse. NP uptake evaluation, employed for understanding how NPs are internalized and trafficked within cells, often employs fluorescence-based assays and flow cytometry. Recent advancements in 3D cell culture models and imaging techniques, such as multiphoton and light-sheet microscopy, have improved the assessment of NP penetration and distribution within more complex 3D structures, offering a more physiologically relevant understanding of NP penetration in dense tumor structures.

Cell viability assessment evaluates the cytotoxic effects of NPs in cancer models, and methodologies are employed to measure cell survival or death. Metabolism-based reagents, such as MTT, WST-1, and resazurin, are widely used for their ability to quantify metabolic activity, though their application in 3D models requires optimization due to limited reagent penetration. MTT, while reliable in 2D cultures, accumulates formazan crystals unevenly in 3D systems and is susceptible to NPs interference. Alternatives like WST-1 and WST-8 offer improved sensitivity and penetration,. ATP-based assays provide rapid and sensitive viability measurements but require cell lysis. Intracellular accumulation-based reagents, such as sulforhodamine B and Neutral Red, measure cell mass or lysosomal retention, respectively, but are less effective in 3D cultures due to diffusion limitations. Plasma membrane integrity-based methods, including LDH release and Live/Dead staining, detect cell death, with Live/Dead assays offering dual-staining of live and dead cells. To address the limitations of 3D structures, advanced approaches such as confocal Z-stack imaging, optimized 3D-specific assays (e.g., CellTiter-Glo® 3D), and extended incubation protocols improve reagent distribution and ensure accurate viability quantification.

Cell death mechanisms—autophagy, apoptosis, and necrosis— can be assessed by various imaging-related methodologies, including transmission electron and fluorescence microscopy, as well as biochemical assays such as Annexin V/propidium iodide staining, acridine orange/ethidium bromide staining, and DNA fragmentation detection via comet assays. Protein markers like caspases and HMGB1 are evaluated using ELISA, western blotting, and flow cytometry to confirm apoptotic and necrotic pathways. However, relying on a single assay can lead to misinterpretation, as some markers may appear in non-cell death contexts, and cancer cell line heterogeneity can affect results. Therefore, combining complementary techniques is necessary to quantify and understand cell death mechanisms induced by nanomaterials accurately.

Investigating cell signaling pathways in response to NP treatment involves a comprehensive analysis of both gene and protein expression. Gene expression analysis, utilizing techniques such as qRT-PCR, DNA microarrays, and RNA sequencing (RNA-seq), provides information about transcriptional changes and helps identify key genes involved in apoptosis, proliferation, and immune response. Protein expression can be assessed by western blotting, immunochemistry, and mass spectrometry. Western blotting is widely used to confirm protein-level changes, such as the downregulation of oncogenes or activation of apoptotic markers, while immunochemistry techniques (ICC and IHC) visualize protein localization in 2D and 3D models. Mass spectrometry-based proteomics offers a broader view of the proteome. Together, these complementary approaches provide a detailed understanding of how NPs influence cellular signaling pathways, aiding in the optimization of nanomedicine for targeted and effective cancer therapy.

The evaluation of oxidative stress is particularly relevant given the dual role of ROS and RNS in cancer, where they can either promote tumor progression or induce cell death depending on their concentration. Oxidative stress arises from an imbalance between ROS/RNS production and the cell’s antioxidant defenses, leading to damage to proteins, lipids, and DNA. Methods for evaluating oxidative stress include direct detection of ROS/RNS using fluorescent probes, measurement of antioxidant defense systems (e.g., glutathione levels and antioxidant enzyme activities), and quantification of oxidative damage biomarkers such as protein carbonylation, lipid peroxidation (e.g., MDA and 4-HNE), and DNA damage (e.g., 8-OHdG). Techniques like LC–MS, western blotting, and immunofluorescence are employed for detecting and localizing oxidative damage.

Cell migration and invasion assays evaluate the metastatic potential of cancer cells and the ability of NPs to inhibit these processes. Various techniques are employed to study these behaviors, including the scratch/wound-healing assay, transwell migration/invasion assay, microfluidic chips, and cell tracking. The scratch assay is a simple and cost-effective method to evaluate wound closure, though it requires antimitotic agents to distinguish migration from proliferation. The transwell assay, utilizing permeable membranes and Matrigel coatings, is widely used to study individual cell migration and invasion. Microfluidic chips provide precise control over experimental conditions, enabling real-time monitoring of cell migration dynamics at the single-cell level, though they require technical expertise. Cell tracking, particularly in 3D models, allows for detailed analysis of migratory trajectories and collective cell behavior, but it demands advanced imaging systems and software for accurate results.

The clonogenic assay assess cell survival and proliferation by measuring the ability of cells to form colonies after exposure to treatments such as ionizing radiation, cytotoxic agents or NPs, avoiding interference from fluorescent or colorimetric indicators. While traditionally performed on cell monolayers, recent advancements have extended its application to 3D models, such as spheroids and microfluidic chips. Studies have demonstrated the assay’s utility in evaluating NP-induced cytotoxicity, with results often showing dose- and time-dependent reductions in colony formation. However, challenges remain in applying the assay to 3D models due to issues like NP penetration and imaging limitations.

Various NPs can interact with cytoskeletal components directly or through drug delivery, leading to structural modifications that impact cell viability and proliferation. Cytoskeletal assessment in in vitro models involves fluorescence microscopy techniques—such as confocal and light sheet microscopy—for high-resolution imaging, biochemical approaches like RT-qPCR and Western blotting for gene and protein expression analysis, and flow cytometry. Biophysical methods such as AFM provide quantitative data on cytoskeletal mechanics, including cell stiffness and elasticity. Light sheet microscopy offers advantages in 3D models by minimizing photobleaching and phototoxicity. Despite advancements in cytoskeleton evaluation, challenges remain, particularly in adapting techniques to dense 3D structures, where imaging depth, reagent penetration, and sample preparation require careful optimization.

Overall, the choice of methodology depends on the specific research question and the complexity of the in vitro model being employed. Combining multiple techniques provides a more comprehensive understanding of NP therapeutic potential, supporting the development of effective nanomedicine-based cancer therapies.

Conclusion

The development of nanocarriers has revolutionized cancer therapy by offering targeted, efficient, and multifunctional delivery of therapeutic agents. By exploring characteristics of the TME, such as the EPR effect, NPs have demonstrated the potential to enhance drug accumulation at tumor sites while minimizing systemic toxicity. However, the clinical translation of nanocarriers remains hindered by limitations in predictive models. Conventional 2D in vitro systems, for example, fail to replicate the complexity of the TME and emphasize the need for adhesion to more physiologically relevant models, such as 3D tumor spheroids, organoids, and microfluidic ToC systems. These advanced models enable high-throughput, cost-effective evaluation of NPs’ therapeutic efficacy and safety. Future efforts in the field of nanomedicine should prioritize optimizing these advanced models to enhance their reproducibility, scalability, and predictive accuracy.

To fully exploit the potential of these advanced models, the selection and optimization of methodologies for evaluating the antitumor efficacy of NPs must be carefully aligned with the cellular processes under investigation and the complexity of the chosen model system. Whether using traditional 2D monolayers or sophisticated 3D systems, a multi-faceted approach is essential, integrating diverse techniques such as metabolic and plasma membrane integrity-based assays, gene and protein expression analysis, and functional tests like migration/invasion and clonogenic assays. This comprehensive strategy ensures that NP-induced effects on cell viability, proliferation, apoptosis, necrosis, autophagy, or oxidative stress are accurately captured. Moreover, these methods offer a more holistic understanding of therapeutic outcomes by reflecting both direct cytotoxicity and secondary effects related to migration, invasion, and intracellular signaling. Achieving accurate and translatable results requires careful adjustments to experimental protocols, including optimizing reagent concentrations, incubation times, and lysis methods—particularly in dense 3D models where diffusion barriers may arise. Cross-validation through complementary techniques, such as combining morphological and biochemical assays for cell death mechanisms or integrating proteomics with transcriptomics for pathway analysis, further enhances data reliability by reducing artifacts and mitigating false interpretations. Ultimately, by refining protocols and embracing multi-technique validation, researchers can generate robust, reproducible results that support the rational design of more effective nanomedicine-based cancer therapies, maximizing their therapeutic impact while minimizing off-target effects.