I've been studying this for a while now. There's a big rabbit hole for you to go into; you can go as deep as you dare. Here's a ok article responding to your question:
https://reolink.com/use-upgrade-existing-home-security-alarm-system/
I have had a wireless system currently at a commercial office for 2 years (DIY install, monitored by a third party) and I have been studying what to do for a residence. The wireless system I have is about 2 years old and has worked great so far. Ironically today I got the first false alarm from the system in that time period.
One of the first things you'll want to decide is if you want to use a wired system or a wireless system, or a combination. Since you already have hard wired sensors installed, it might be worth keeping some or maybe all of them. If you have good, hidden wired door and window sensors, I would reuse those.
The wireless door and window sensors are very ugly and very visible. You'll also need to decide if you want to go DYI or professional install and if you want to self monitor or use a central station.
If you have interest in selecting and putting together your own system (panel, door and window sensors, motion detectors, cameras, smoke and fire detectors, etc), I suggest checking out alarmgrid.com and geoarm.com (don't recommend ordering from geoarm, as I just had a bad experience with them, but their web site has a lot of information on a lot of products). You should surf their sites, and either of them provide tons of information about their products and monitoring services and they help without much if any sales pressure. These guys seem to concentrate primarily on wireless systems, but you can configure either a wired or wireless or combination systems with them (you have to get a device that translates the hardwired signals to wireless). If you get monitoring through them (they do not have any long term contract requirements), they use the Alarm.com monitoring and phone app system, which is the same as Frontpoint (below) uses.
If you want a pre-curated DYI system where someone has done most the brain damage for you might look at Frontpoint.com or Simplisafe.com, for example. These two are wireless only set ups (as far as I know). These guys mostly let you pick from a limited, preselected menu of sensors, panels, etc. They tend to be a bit obscure as to what exactly you are getting until you ask specifically, but frontpoint has great customer support (I've not tried simplisafe). Frontpoint does require a monitoring contract. Simplisafe does not appear to. The one positive with both these guys is that it is a pretty turn key set up, other than having to install it yourself, so they limit what sensors, sensor types, etc. you can choose from. As far as I know, both of these are wireless equipment only.
If you want to keep your entire wired system, there doesn't seem to be a lot of DYI options. In fact, I wasn't able to find any; it'd be great if someone can find one and let us know.