Crowding, the inability to recognize an individual object in clutter (Bouma, 1970), is considered a major impediment to object recognition in peripheral vision. Despite its significance, the cortical loci of crowding are not well understood. In particular, the role of the primary visual cortex (V1) remains unclear. Here, we utilize a diagnostic feature of crowding to identify the earliest cortical locus of crowding. Controlling for other factors, radially arranged flankers induce more crowding than tangentially arranged ones (Toet and Levi, 1992). We used functional magnetic resonance imaging (fMRI) to measure the change in mean blood-oxygenation-level-dependent (BOLD) response due to the addition of a middle letter between a pair of radially or tangentially arranged flankers. Consistent with the previous finding that crowding was associated with a reduced BOLD response (Millin et al, 2013), we found that the BOLD signal evoked by the middle letter depended on the arrangement of the flankers: less BOLD response was associated with adding the middle letter between radially arranged flankers as compared to adding it between tangentially arranged ones. This anisotropy in BOLD response was present as early as in V1 and remained significant in downstream areas. The effect was observed while subjects' attention was diverted away from the testing stimuli. Contrast detection threshold for the middle letter was unaffected by flanker arrangement, ruling out surround suppression of contrast response as a major factor in the observed BOLD anisotropy. Our findings support the view that V1 contributes to crowding.