All the private property, including the Panemunė Castle, was seized by the Tsarist government. Left without owners, the castle saw little maintenance performed by a series of administrators and in time became increasingly decrepit.
The Panemunė Castle passed into the hands of the Vilnius Academy of Arts in 1982. Now the castle houses a museum, exhibition spaces and tourist information centre. The redesigned castle court is used as a venue for various cultural activities: musical events, festivals, traditional craft fairs.
On the right bank of the Nemunas river is one of the most beautiful Renaissance era building in Lithuania. The castle architecture features elements of the baroque and neo-classical styles, as well.
In 1597, a noblemen and timber merchant Janusz Eperyasz of Hungarian origin acquired the Panemunė manor house. The new residence in the style of defensive castle most probably was built by the Dutch architect Peter Nonhaardt.
Construction was started in 1604 and completed in 1610. The Eperyasz family owned the Panemunė Castle for 158 years.
In 1759 the Panemunė Castle was purchased by Antanas Gedgaudas, the head of Akmenė and standard-bearer of the Grand Duchy of Lithuania. The castle was transformed into a lavish and luxurious residence of the Gedgaudas family.
The architecture of the restored castle acquired neo-classical features. This style was employed in the re-cultivation of the ponds and in the design of the park.
The castle ceased to be inhabited after the 1831 revolt against Russian imperial regime led by General Anthony Gielgud who was killed while fighting against the Russian army.