Was I right in not joining military?

I've never lived outside of Texas. Being young I always considered joining the united states military. My grandfather a WWII veteran, father a Vietnam veteran and me, could've been Iraq veteran. I moved to Indiana and was about to enlist, when I noticed how people(white)here would look down on me because I was Latino. No matter if I spoke perfect English, had an Associates Degree in Criminal Justice or loved hearing Country music, to them I was still nothing more than an illegal that had to crossed The border, faking to be American. I made friends, and once they got to know me really well, after telling them I was born and raised in Texas, they still joked around with immigrant jokes and I realized no matter what, to them I'm an illegal.. They won't accept me and that's when I called my recruiter and told him I wasnt fighting for a racial country.