You are indeed correct. When you work for a firm, the firm holds the copyrights in your work under the concept that the work was a "work for hire". If you use the material without permission, you could be in violation of the copyright. While the damages might be limited, in concept you would still be subject potentially to a lawsuit, and if your employer registered the copyright prior to your infringement then despite the relatively limited damages you might be subject to, the copyright holder may also be entitled to what are called "statutory damages" (statutory damages are an exponentially larger sum the court could impose to punish an intentional infringement where the actual damages are limited - simply because we as a society wish to punish infringement). The scenario you describe would be an intentional infringement.
Understand also the copyright statute allows the prevailing party to recover attorney fees.
The AIA canons of ethics address this question and suggest that you should be permitted to copy materials for use in your own portfolio. However, in this instance it appears you wish to continue your employment while using the work of your employer (and while you did perform the work, it is the work of your employer) to build what at some point could be a competing business. That could also implicate truth in advertising claims and even trade secret claims depending on how the facts were to develop. That isn't a risk you really want to run - particularly not if you do intend to start your own practice someday.
Suffice it to way I would not recommend simply doing it and hoping you are not caught. I would recommend that if you feel strongly about it you consider approaching your employer and explaining your intentions. If your employer agrees to let you use the materials there of course will be no issue. If your employer doesn't agree to let you use the materials then you know they object to their use and would likely seek to punish any such use. If for some reason you feel you can't talk to your employer about it, that likely answers your question too. If you can't address it with them openly (and there would always be the risk they would find the website if you simply proceeded without their permission and hoped for the best) then you likely know what their answer would be if you did.
It may be unfortunate but that is the reality. While responsible firms will allow you to use materials for personal portfolios (i.e.; to seek new employment) there really isn't anything that requires them to allow you to use it solicit clients or build what could become a competing business. That may not seem fair, but it is understandable when you consider it from the firm's perspective.
-------------------------------------------
Frederick Butters FAIA
Attorney
Southfield MI
-------------------------------------------