The title is capitalised in a job advert, but not in conversation.
Say 'I am working as an Oracle developer' if you are one of several. If you are the sole holder of that position, say 'I am working as Oracle developer for Delphi Prognostications, Inc.'
If you think of the context, "Oracle Developer" is not a proper definition. It seems you are a DBA, not a Oracle Developer. "I am a Oracle DBA" is a much more proper definition. You develop something over framework or language, not over a corporation.
What is a 'DBA'?
If it's a job title. it's not a definition.
Your second sentence makes little sense to me.
So an Oracle Developer develops [an] Oracle, not for Oracle. Is that right?
Oracle is a Corporation. As a corporation, it creates or buys some frameworks or languages in order to maintain its product, or simply to profit. Some of the languages presented by Oracle Corporation are "PL/SQL" , "Java" to name a few. If you develop something over Java language, you are a Java Developer, not Oracle Developer. Likewise if you develop something for your PC that have Windows operating system, you will be C Sharp developer (name of language) or WPF developer, (name of framework) not a Microsoft Developer."software developer on Oracle stack" would be a proper definiton. It would be a technical issue, so it could be a bit confusing.