It seems like you already have a good idea of where to start, plus a point of reference, so my advice is going to be short.
- Keep your magic system straightforward and stick to your world’s “laws of magic.”
- If you are drawing inspiration for your magic system from other cultures, be respectful.
- Keep your lore in line. Unless it is a plot point in your story, make sure that there aren’t any unintentional loopholes or contradicting notions.
- Try researching urban fantasy and read some samples from books in that genre. That may help spark some inspiration as to how magic/non-magic persons live in modern society.
- Don’t use magic as an allegory for a repressed people.
One of the reasons why I’m purposely trying to keep this post short is because fantasy is a genre that is completely unique to the writer. I don’t want to put words in your mouth or different ideas in your head and possibly throw you off your original track because to be completely honest, you don’t have to take anyone’s advice about your world. Since you’re writing a fantasy story, there is no limit as to how you can develop your plot. Anything is possible as long as it can be imagined.