Spectres spirit reservation.

Each time when i go on map or leave it (or sometimes even on map) - all my spectres disappears. And skill says u dont have spirit. but i have free spirit. Almost impossible play now. :(
That fis if i dpor spectre gem in invetory and put it back. But it is so canser - all put all it links back.