The tumor microenvironment (TME) plays a central role in cancer development and progression. It represents a complex network of cancer cell (sub-)clones and a variety of stromal cell types. Recently, new technology platforms shed light on the cellular composition of the TME at very high resolution and identified a complex landscape of multi-lineage immune cells (e.g., T and B lymphocytes, myeloid cells, and dendritic cells), cancer associated fibroblasts (CAF) and tumor endothelial cells (TECs). A growing body of evidence suggests that metabolically, genetically and on their transcriptomic profile TECs exhibit unique phenotypic and functional characteristics when compared to normal endothelial cells (NECs). Furthermore, the functional role of TECs is multifaceted as they are not only relevant for promoting tumor angiogenesis but have also evolved as key mediators of immune regulation in the TME. Regulatory mechanisms are complex and profoundly impact peripheral immune cell trafficking into the tumor compartment by acting as major gatekeepers of cellular transmigration. Moreover, TECs are associated with T cell priming, activation and proliferation by acting as antigen-presenting cells themselves. TECs are also essential for the formation of tertiary lymphoid structures (TLS) within the tumor, which have recently been associated with treatment response to checkpoint antibody therapy. Further essential characteristics of TECs compared to NECs are their high proliferative potential as well as greatly altered gene expression profile (e.g., upregulation of pro-angiogenic, extracellular matrix remodeling, and stemness genes), which results in enhanced secretion of immunomodulatory cytokines and altered cell-surface receptors [e.g., major histocompatibility complex (MHC) and immune checkpoints]. The TEC phenotype may be rooted in an aggressive tumor micro-milieu based on cellular stress via hypoxia and reactive oxygen species (ROS). Vice versa TECs might modulate TME immunogenicity thereby fostering cancer-associated immune suppression. This review aims to elucidate the currently emergent pathophysiological aspects of TECs with a particular focus on their potential role as regulators of immune cell function in the TME. It is a main future challenge to deeply characterize the phenotypic and functional profile of TECs to illuminate their complex role within the TME. The ultimate goal is the identification of TEC-specific drug targets to improve cancer (immuno-)therapy.