Try both, stick with what works for you.
I use OpenGL, so the only advice I can give pertains to that.
If you want to get started with OpenGL (you probably don't), you want to first learn some math. Learn about matrix algebra particularly, and then move on to actually learning it.
Resources for learning modern lower-level graphics programming are shit. Everything from loading up the buffers to shader programs, etc. It's all shit for beginners. If you do OpenGL, start with an older edition of The OpenGL Programming Guide (like one targeting an OpenGL version 2.x), and actually learn the deprecated, dead fixed-function pipeline, even with immediate mode. You will learn nothing that you will directly use with modern graphics programming, but you will build the foundation necessary to move up.
Then add working with VBOs, then add VAOs, then get a basic shader program working, and you're well on your way.