When we talk about portfolio lenders it commonly corresponds to Savings & Loan institutions, and sometimes banks.  They are called portfolio lenders on the ground that they tend to originate loans for their own portfolio (usually adjustable rate loans), not for resale in the secondary market.  But in the recent years the distinction became less clear because the majority of portfolio lenders also engaged in mortgage banking.

          Contrary to other lenders they are likely to pay more compensation to their loan officers for originating a portfolio product than for originating a fixed rate loan. In your search for a mortgage it is widely considered that they are not as competitive as mortgage bankers and brokers in the fixed rate loan market. However portfolio lenders made large efforts to improve their competitiveness.
Portfolio lenders are often considered as a "second resort" for those who cannot qualify for a fixed rate loan. Why?  In the case your loan officer is steering you towards "sub-prime" loans, you might be well advised to check out a portfolio lender first.

          These lenders also can serve as "niche" lenders because certain things are more important to them than meeting the more standardized underwriting guidelines of a mortgage banker. An example would be a savings & loan company which grant more relevance to an individual's savings history than being able to fully document income.

          Eventually the last thing you need to know is that if you apply for a loan with a portfolio lender and you are declined, that's it for you. It is very unlikely for you to get a loan from another lender. But if you still think you should be able to qualify for a loan, you have to start over somewhere else.