In this review, we highlight the complexities of the natural history, biology, and clinical management of three intermediate connective tissue tumors: desmoid tumor (DT) or aggressive fibromatosis, tenosynovial giant cell tumor (TGCT) or diffuse-type pigmented villonodular synovitis (dtPVNS), and giant cell tumor of bone (GCTB). Intermediate histologies include tumors of both soft tissue and bone origin and are locally aggressive and rarely metastatic. Some common aspects to these tumors are that they can be locally infiltrative and/or impinge on critical organs, which leads to disfigurement, pain, loss of function and mobility, neurovascular compromise, and occasionally life-threatening consequences, such as mesenteric, bowel, ureteral, and/or bladder obstruction. DT, PVNS, and GCTB have few and recurrent molecular aberrations but, paradoxically, can have variable natural histories. A multidisciplinary approach is recommended for optimal management. In DT and PVNS, a course of observation may be appropriate, and any intervention should be guided by symptoms and/or disease progression. A surgical approach should take into consideration the infiltrative nature, difficulty in obtaining wide margins, high recurrence rates, acute and chronic surgical morbidities, and impact on quality of life. There are similar concerns with radiation, which especially relate to optimal field and transformation to high-grade radiation-associated sarcomas. Systemic therapies must be considered carefully in light of acute and chronic toxicities. Although standard and novel therapies are promising, many unanswered questions, such as duration of therapy and optimal end points to evaluate efficacy of drugs in clinical practice and trials, exist. Predictive biomarkers and novel clinical trial end points, such as volumetric measurement, magnetic resonance imaging T2 weighted mapping, nuclear imaging, and patient-reported outcomes, are in development and will require validation in prospective trials.