In the valuation of properties with outstanding intangible (cultural) value, it is thought that cultural value, having unique (nonmaterial) features, cannot be assessed by the application of market valuation (e.g., in mass valuation). However, the 2012 International Valuation Standards (IVSC 2012) contain guidance (requirements for and considerations of valuation approaches) for valuing “the interests of historic real property.”Nevertheless, disputable issues remain: What criteria should be used for reconciliation of the final opinion on value and to what degree? Does the assessed value represent the market value of older (unique) construction? Can special value (probably cultural value for buyers or owners) also be a valuation attribute (e.g., value reconsideration factor) and how (to what degree) can it be measured? This theoretical study discusses two types (concepts) of values affecting architectural heritage buildings—cultural value and market value—and assessment principles, in which the feature cultural heritage can be taken into account. The Historical Centre of Riga, (HCR), Latvia, and its protective zone (PZ) were selected for characterizing urban shrinkage and the loss of livability in historical cities.