Hmm, I posted a big reply to this earlier, and it didn't appear.
Oh well, basically, there are two types of schools: the big chains and the small independent schools. I'm not sure how much you know about this, so apologies if I'm telling you things you already know (I've looked into this a lot and had 2 unsuccessful interviews so far).
The big chain schools (Nova, GEOS, AEON, WinBe, ECC and Berlitz) are conversation schools, and are generally more about showing students a good time and having fun lessons, rather than teaching them effectively (as long as they keep paying, they're not bother how good you are). They tend to employ more based on your personality than your teaching ability. A CELTA or equivalent won't get you a job with one of them, but for someone with no exprience, it can certainly make you more confident in front of a class. If you're a naturally outgoing and confident person however, it can wait until you have a few years experience with one of these schools. Just a few other things to consider. Generally, these schools won't hire from inside Japan (I think Berlitz will though). They require 12 years of formal education in English language. They require a degree in any subject. They want you to sign a 12 month contract, and to be able to go anywhere in the country you're needed, which may be an issue if you're already living in Japan and were looking for somewhere local. I think there may also be some visa issues for you, so do a bit of research into that (something to do with their company having to sponsor your visa, which is why they require a 12-month contract). Quick tip: If you complete a year with GEOS, they will contribute half of the course fees for a CELTA in London or New Zealand. In these chain schools, you usually get asked at the interview whether you would like to teach adults or children (everyone has to teach all ages, but you can specialise to some degree). Some of them also have specialised kids schools that you can apply to seperately. All of the above schools have extensive websites, so if you're interested, Google them and take a look.
As for the smaller independent schools, I haven't really looked into them much, but obviously you pick up a bit in conversation with other teachers. From what I can tell, the initial job will be as an assistant to a qualified teacher, and can be any range of responsiblities from a glorified CD player to actually taking the lesson yourself. But the large chain schools have a poor reputation, and as such, jobs in independent schools tend to be fairly competitive. Most people go into this after working with the chain schools for a couple of years, once they've established themselves in Japan. So you'll be competing not only with people with CELTAs, but also people with a few years experience. Don't let that put you off, but be aware of it. Most people use the chain schools to get into Japan in the first place, so I would say use the fact that you're already there to miss this step out and go straight to the independents. But bear in mind that if you're planning on making a career out of it, you will need to get some qualifications at some point, because the salary for an unqualified teacher in Japan is quite low. But the independent sector sounds quite exciting for an experienced teacher, since you get to do things like designing the entire curriculum.
Anyway, I found this website really useful. Also Gaijinpot and Metropolis both have forums with teaching sections, where there are plenty of experienced and new teachers there to offer advice about any part of the process of getting a job teaching. Good luck.