The roots of story lies in the Satyuga when Ram was in exile. Sugreev requested Lord Rama’s help in 
defeating Bali who had wrongfully captured his wife.

So lord Rama advised Sugreev to call Bali for a hand to hand combat.
 

when Lord Ram hid himself behind a tree and when Bali and Sugreev fight, He shot an arrow killing Bali. Bali didn’t die immediately he spoke with Lord Ram as the Lord kept his head on his lap.           

 The conversation went like this:

 

 Bali: Lord I know that you have a task to find out your wife and you killed me in order to gain friendship with Sugreev.

 
Lord Ram: Yes Bali, it’s time for you. I however feel guilty of killing you and I want to give you the power of a boon in retaliation of the overall scene.
 
Bali: Who has the amazing fate like I have? To die in your hands and in your lap? I will attain moksha anyways!
 
Lord Ram: I don’t keep anyone’s debt my dear.If you feel that I should be guilty, you can curse me either. I will accept all of it with dignity.
 
Bali: Yes I feel that you killed me in unfair war. By hiding behind a tree. By pulling an arrow with those fingers. I have 2 wishes: 1. May both your palms vanish! 2. Since I am the king of this area and I took care of each living creature like a father, the tree who sheltered you from my sight is equally guilty of killing me. May he become a bitter tree, so bitter that nobody eats any part without complaining of its bitterness.
 
Lord Ram: I can’t give that to you in this Yuga. May it come true in Kaliyuga!
 
Bali died peacefully.
 
Then when Lord Ram started to go back, the tree cried it’s heart in front of Him saying “Is this my curse in return for my devotion towards You my Lord? I served you and now people will insult me in return?”
 
Lord Ram held the tree and said, “Curses are again Boons sometimes! In Kaliyuga when people will keep calling you as Bitter Neem Tree, like today you sheltered me, I will shelter you by making my own Body from your trunk. And yes like promised to Bali, I will not have palms or fingers, I will end my hands at my wrist!”
 
That’s why Lord Jagannath doesn’t have palms or fingers and is made up of Neem wood! He kept his promise to Bali and the tree both!
 
Jai Jagannath!!!