The problem with the Caeser cipher technique is there are only 25 possible keys to decrypt the ciphertext. So, if someone performs a brute force attack on Caesar cipher technique all he has to do is try all possible keys from 1 to 25. one of those keys will give you the original message.

Full Explanation:

Caesar cipher technique works in such a way that all the letters in the original text are replaced by letters some k places further down in the alphabet. 
If we give alphabets a number such as...

 a   b   c   d   e   f  g   h  i    j     k    l     m    n    o    p    q    r     s   t     u     v    w    x    y    z
 1   2   3   4   5  6  7  8  9  10  11  12  13   14  15  16  17  18  19  20  21  22  23  24   25 26

if k = 3; three places further down the letter 'a' is 'e'.

We can express the caesar cipher algorithm in 
C = E(k, p) = (p + k) mod 26
 Where C is ciphertext
 E is Encryption
 k is key. ( can be any number from 1 - 25)
 p is plaintext.

 The decryption algorithm is
 p = D(k, C) = (C - k) mod 26 

by trying all possible values k can take, an attacker can easily deduce the key and plain text

plaintext: wedge shaped ball whirling up in the wool
ciphertext: zhgjh vkdshg edoo zkluolqj lq wkh zrro (for k = 3)

So in brief, these three points enable the attacker to use brute force cryptanalysis on the said algorithm.

1. The encryption and decryption algorithms are known.
2. There are only 25 keys to try.
3. The language of the plaintext is known and easily recognizable.