While what you paint on is highly dependent on what type of paint you use, beginners should consider substrates that require the least amount of prep work. Canvas board, and prestretched canvas are options that are readily available, affordable, and in most cases, ready to use right out of the wrapping. Illustration board is another option if you're using watercolors or acrylics.
Each painting medium has different strengths and weaknesses that must be considered. Watercolors are easy to use and maintain, but their translucent nature can be a challenge for beginners. Acrylics are more opaque, but are harder to mix and maintain. They can, however, lead to more realistic results. Oil paints are the easiest to blend and can be used to to create a smooth, lush look in your painting, but have to be mixed with turpentine or another oil medium and take a long time to dry.
Whether you get soft- or hard-bristled brushes depends on your painting style and the types of paint you are using. Thicker paints require stiffer brushes and thinner paints, like watercolor, require softer brushes. Buy round and flat brushes in three sizes of each type. Buy a fan brush for blending and a large brush for applying large areas of paint.
An often overlooked item is the one on which paints are mixed. An inexpensive option is palette paper, which is nothing more than sheets of wax paper that do not absorb paint. While plastic and wood palettes are a popular option, they can be difficult to clean. You can also use a large piece of glass. Its an inexpensive option that works well.
Consider buying masking tape to block off borders. Frisket or masking fluid serve the same purpose. A good easel or drawing table can make it easier to work. Also consider a painting stick, which you can position a few inches above the canvas to give you something to rest your hand on while you paint.