tdf#121441 improve DOCX footnote import

ODF represents footnotes by using a fixed string for the label
(text:note-citation) and a flexible body (text:note-body) for the
representation in the footnote area. The only formatting of the
footnote reference is done by changing the character class assigned
to the anchor (which is the text range of the label in the text).
For most of the setting, the footnote area label just follows the
footnote body character formatting.

OTOH MS Word has no such "restrictions". It handles the label just
as concated, formated text runs with the same style. On top of it,
DOCX completely splits the reference from the footnote area part,
including its own label, which can easily result in completely
different labels for the footnote and the reference, as I happened
to repoduce for my test documents.

At this point it's quite obvious that for any complex footnotes,
LibreOffice won't be able to represent them. IMHO ODF should offer
the same flexibility for the label and the body and allow all the
normal formatting in the label. I'm not sure that getting footnote
area and reference label out of sync is a good idea.

So this patch tries to improve the situation in the current
constraints set by ODF.

1. It imports all runs of the whole custom DOCX footnote label.
2. If any run contains a symbol, switches the font of the whole
label to the referenced symbol font.
3. Completely ignores the label of the footnote area and overrides
the font of the footnote area label with the font of the
reference.

Other problems I found while testing this code:

1. LO edit field correctly gets the font and character set, but
displays empty glyphs. So no real way to edit the label.
2. Normally the font of the footnote area label would follow the
footnote font. This doesn't work anymore when the font is
overridden for the label. Setting the whole font of the label
to Symbol doesn't seem like a good solution either.
3. You can't mix multiple fonts, or even symbols and letters, as
you can just select one font for the label.
4. You can't change the footnote are label font at all and since
it doesn't follow the footnote area anymore, there is basically
no way to change it.

Change-Id: Iafa16936be81e1866c610ebf0f71ab15e74dd059
Reviewed-on: https://gerrit.libreoffice.org/81370
Tested-by: Jenkins
Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>

4 files changed, 140 insertions, 16 deletions

diff --git a/writerfilter/source/dmapper/DomainMapper.cxx b/writerfilter/source/dmapper/DomainMapper.cxx
index 5ad13d6145e5..50c546a31e0f 100644
--- a/writerfilter/source/dmapper/DomainMapper.cxx
+++ b/writerfilter/source/dmapper/DomainMapper.cxx
@@ -80,6 +80,9 @@
 #include <dmapper/GraphicZOrderHelper.hxx>
 #include <tools/diagnose_ex.h>
 #include <sal/log.hxx>
+#include <vcl/svapp.hxx>
+#include <vcl/outdev.hxx>
+#include <vcl/font.hxx>
 
 using namespace ::com::sun::star;
 using namespace oox;
@@ -945,10 +948,11 @@ void DomainMapper::lcl_attribute(Id nName, Value & val)
         }
         break;
         case NS_ooxml::LN_CT_FtnEdnRef_customMarkFollows:
-            m_pImpl->SetCustomFtnMark( true );
+            m_pImpl->StartCustomFootnote(m_pImpl->GetTopContext());
         break;
         case NS_ooxml::LN_CT_FtnEdnRef_id:
             // footnote or endnote reference id - not needed
+            m_pImpl->StartCustomFootnote(m_pImpl->GetTopContext());
         break;
         case NS_ooxml::LN_CT_Color_themeColor:
             m_pImpl->appendGrabBag(m_pImpl->m_aSubInteropGrabBag, "themeColor", TDefTableHandler::getThemeColorTypeString(nIntValue));
@@ -2139,6 +2143,9 @@ void DomainMapper::sprmWithProps( Sprm& rSprm, const PropertyMapPtr& rContext )
     case NS_ooxml::LN_EG_RPrBase_rStyle:
         {
             OUString sConvertedName( m_pImpl->GetStyleSheetTable()->ConvertStyleName( sStringValue, true ) );
+            if (m_pImpl->CheckFootnoteStyle())
+                m_pImpl->SetHasFootnoteStyle(m_pImpl->GetFootnoteContext()->GetFootnoteStyle() == sConvertedName);
+
             // First check if the style exists in the document.
             StyleSheetEntryPtr pEntry = m_pImpl->GetStyleSheetTable( )->FindStyleSheetByConvertedStyleName( sConvertedName );
             bool bExists = pEntry.get( ) && ( pEntry->nStyleTypeCode == STYLE_TYPE_CHAR );
@@ -2675,11 +2682,38 @@ void DomainMapper::sprmWithProps( Sprm& rSprm, const PropertyMapPtr& rContext )
         resolveSprmProps(*this, rSprm);
         SymbolData  aSymbolData = m_pImpl->GetSymbolData();
         uno::Any    aVal = uno::makeAny( aSymbolData.sFont );
-        if( rContext->GetFootnote().is())
+        auto xFootnote = rContext->GetFootnote();
+        if (!xFootnote.is() && m_pImpl->IsInCustomFootnote())
+            xFootnote = m_pImpl->GetFootnoteContext()->GetFootnote();
+        if (xFootnote.is())
         {
-            uno::Reference< beans::XPropertySet > xAnchorProps( rContext->GetFootnote()->getAnchor(), uno::UNO_QUERY );
-            xAnchorProps->setPropertyValue( getPropertyName( PROP_CHAR_FONT_NAME), aVal);
-            rContext->GetFootnote()->setLabel(OUString( aSymbolData.cSymbol ));
+            // DOCX can have different labels for the footnote reference and the footnote area.
+            // This skips the one from the footnote area and just uses the reference one.
+            if (!m_pImpl->IsInFootOrEndnote())
+            {
+                auto xAnchorRange = xFootnote->getAnchor();
+                auto xAnchorCursor(xAnchorRange->getText()->createTextCursorByRange(xAnchorRange));
+
+                // append a dummy character, so the following properties will be set as
+                // as SwpHints::SwTextAttr instead of the SwAttrSet of the paragraph,
+                // which would be removed by SwXText::Impl::finishOrAppendParagraph
+                xAnchorCursor->collapseToEnd();
+                uno::Reference<text::XTextRange> xHackRange(xAnchorCursor, uno::UNO_QUERY);
+                xHackRange->setString("x");
+
+                uno::Reference<beans::XPropertySet> xAnchorProps(xAnchorRange, uno::UNO_QUERY);
+                xAnchorProps->setPropertyValue(getPropertyName(PROP_CHAR_FONT_NAME), aVal);
+                xAnchorProps->setPropertyValue(getPropertyName(PROP_CHAR_FONT_NAME_ASIAN), aVal);
+                xAnchorProps->setPropertyValue(getPropertyName(PROP_CHAR_FONT_NAME_COMPLEX), aVal);
+                xAnchorProps->setPropertyValue(getPropertyName(PROP_CHAR_FONT_CHAR_SET), uno::makeAny(awt::CharSet::SYMBOL));
+
+                // remove the dummy char
+                xHackRange->setString("");
+
+                OUString sLabel = xFootnote->getLabel();
+                sLabel += OUStringChar(aSymbolData.cSymbol);
+                xFootnote->setLabel(sLabel);
+            }
         }
         else //it's a _real_ symbol
         {
@@ -3032,6 +3066,11 @@ void DomainMapper::lcl_startCharacterGroup()
 
 void DomainMapper::lcl_endCharacterGroup()
 {
+    if (m_pImpl->CheckFootnoteStyle())
+    {
+        m_pImpl->SetCheckFootnoteStyle(m_pImpl->IsInCustomFootnote());
+        m_pImpl->SetHasFootnoteStyle(false);
+    }
     m_pImpl->PopProperties(CONTEXT_CHARACTER);
 }
 
@@ -3105,10 +3144,9 @@ void DomainMapper::lcl_text(const sal_uInt8 * data_, size_t len)
             m_pImpl->clearDeferredBreaks();
         }
 
-        if( pContext->GetFootnote().is() && m_pImpl->IsCustomFtnMark() )
+        if( pContext->GetFootnote().is() && m_pImpl->IsInCustomFootnote() )
         {
-            pContext->GetFootnote()->setLabel( sText );
-            m_pImpl->SetCustomFtnMark( false );
+            pContext->GetFootnote()->setLabel(sText);
             //otherwise ignore sText
         }
         else if (m_pImpl->IsOpenFieldCommand() && !m_pImpl->IsForceGenericFields())
@@ -3164,6 +3202,9 @@ void DomainMapper::lcl_positivePercentage(const OUString& rText)
 
 void DomainMapper::lcl_utext(const sal_uInt8 * data_, size_t len)
 {
+    // All these fixed values are defined as static const sal_Unicode codepoints in the fast parser,
+    // like uFtnEdnRef = 0x2, uFtnEdnSep = 0x3, ... and have a len of 1, if they aren't valid unicode.
+
     OUString sText(reinterpret_cast<const sal_Unicode *>(data_), len);
     const RubyInfo & aInfo = m_pImpl->GetRubyInfo();
     if (aInfo.nSprmId == NS_ooxml::LN_CT_Ruby_rt)
@@ -3175,6 +3216,44 @@ void DomainMapper::lcl_utext(const sal_uInt8 * data_, size_t len)
         return;
     }
 
+    if (len == 1)
+    {
+        // If the footnote contains a Footnote Reference Mark, it can't be a custom footnote
+        // ******
+        // This code block is wrong, as it should also be in m_pImpl->IsInFootOrEndnote().
+        // The main problem is that
+        //
+        //  assert(len != 1 || sText[0] != 0x2)
+        //
+        // is triggered by the unit test SwLayoutWriter::testForcepoint75, so all these pseudo
+        // value handling is broken.
+        // But this is just a symptom, as I guess it's possible to generate broken DOCX documents,
+        // which might be problematic, triggering *funny* code paths left and right.
+        // ******
+        if (sText[0] == 0x2)
+        {
+            m_pImpl->EndCustomFootnote();
+            return;
+        }
+
+        if (m_pImpl->IsInCustomFootnote())
+        {
+            if (sText[0] != 0xd && sText[0] != 0x3)
+            {
+                // DOCX can have different labels for the footnote reference and the footnote area.
+                // This skips the one from the footnote area and just uses the reference one.
+                if (!m_pImpl->IsInFootOrEndnote())
+                {
+                    auto xFootnote = m_pImpl->GetFootnoteContext()->GetFootnote();
+                    xFootnote->setLabel(xFootnote->getLabel() + sText);
+                }
+                return;
+            }
+            else
+                m_pImpl->SetHasFootnoteStyle(true);
+        }
+    }
+
     if (m_pImpl->isSdtEndDeferred())
     {
         // In case we have a field context, then save the property there, so
diff --git a/writerfilter/source/dmapper/DomainMapper_Impl.cxx b/writerfilter/source/dmapper/DomainMapper_Impl.cxx
index db561162cd24..d41330700b0c 100644
--- a/writerfilter/source/dmapper/DomainMapper_Impl.cxx
+++ b/writerfilter/source/dmapper/DomainMapper_Impl.cxx
@@ -219,10 +219,11 @@ DomainMapper_Impl::DomainMapper_Impl(
         m_eInHeaderFooterImport( HeaderFooterImportState::none ),
         m_bDiscardHeaderFooter( false ),
         m_bInFootOrEndnote(false),
+        m_bHasFootnoteStyle(false),
+        m_bCheckFootnoteStyle(false),
         m_bSeenFootOrEndnoteSeparator(false),
         m_bLineNumberingSet( false ),
         m_bIsInFootnoteProperties( false ),
-        m_bIsCustomFtnMark( false ),
         m_bIsParaMarkerChange( false ),
         m_bParaChanged( false ),
         m_bIsFirstParaInSection( true ),
@@ -522,7 +523,7 @@ bool DomainMapper_Impl::GetIsFirstParagraphInSection( bool bAfterRedline ) const
     return ( bAfterRedline ? m_bIsFirstParaInSectionAfterRedline : m_bIsFirstParaInSection )
                 && !IsInShape()
                 && !m_bIsInComments
-                && !m_bInFootOrEndnote;
+                && !IsInFootOrEndnote();
 }
 
 void DomainMapper_Impl::SetIsFirstParagraphInShape(bool bIsFirst)
@@ -2126,6 +2127,7 @@ void DomainMapper_Impl::PopPageHeaderFooter()
 
 void DomainMapper_Impl::PushFootOrEndnote( bool bIsFootnote )
 {
+    assert(!m_bInFootOrEndnote);
     m_bInFootOrEndnote = true;
     m_bCheckFirstFootnoteTab = true;
     m_bSaveFirstParagraphInCell = m_bFirstParagraphInCell;
@@ -2134,7 +2136,13 @@ void DomainMapper_Impl::PushFootOrEndnote( bool bIsFootnote )
         // Redlines outside the footnote should not affect footnote content
         m_aRedlines.push(std::vector< RedlineParamsPtr >());
 
+        // IMHO character styles from footnote labels should be ignored in the edit view of Writer.
+        // This adds a hack on top of the following hack to save the style name in the context.
         PropertyMapPtr pTopContext = GetTopContext();
+        OUString sFootnoteCharStyleName;
+        boost::optional< PropertyMap::Property > aProp = pTopContext->getProperty(PROP_CHAR_STYLE_NAME);
+        if (aProp)
+            aProp->second >>= sFootnoteCharStyleName;
 
         // Remove style reference, if any. This reference did appear here as a side effect of tdf#43017
         // Seems it is not required by LO, but causes side effects during editing. So remove it
@@ -2148,7 +2156,7 @@ void DomainMapper_Impl::PushFootOrEndnote( bool bIsFootnote )
                 OUString( "com.sun.star.text.Footnote" ) : OUString( "com.sun.star.text.Endnote" )),
             uno::UNO_QUERY_THROW );
         uno::Reference< text::XFootnote > xFootnote( xFootnoteText, uno::UNO_QUERY_THROW );
-        pTopContext->SetFootnote( xFootnote );
+        pTopContext->SetFootnote(xFootnote, sFootnoteCharStyleName);
         uno::Sequence< beans::PropertyValue > aFontProperties = pTopContext->GetPropertyValues();
         appendTextContent( uno::Reference< text::XTextContent >( xFootnoteText, uno::UNO_QUERY_THROW ), aFontProperties );
         m_aTextAppendStack.push(TextAppendContext(uno::Reference< text::XTextAppend >( xFootnoteText, uno::UNO_QUERY_THROW ),
@@ -2160,6 +2168,11 @@ void DomainMapper_Impl::PushFootOrEndnote( bool bIsFootnote )
         CheckRedline( xFootnote->getAnchor( ) );
         m_aRedlines.push( aFootnoteRedline );
 
+        // Try scanning for custom footnote labels
+        if (!sFootnoteCharStyleName.isEmpty())
+            StartCustomFootnote(pTopContext);
+        else
+            EndCustomFootnote();
     }
     catch( const uno::Exception& )
     {
@@ -2289,6 +2302,22 @@ void DomainMapper_Impl::EndParaMarkerChange( )
     m_currentRedline.clear();
 }
 
+void DomainMapper_Impl::StartCustomFootnote(const PropertyMapPtr pContext)
+{
+    if (pContext == m_pFootnoteContext)
+        return;
+
+    assert(pContext->GetFootnote().is());
+    m_bHasFootnoteStyle = true;
+    m_bCheckFootnoteStyle = !pContext->GetFootnoteStyle().isEmpty();
+    m_pFootnoteContext = pContext;
+}
+
+void DomainMapper_Impl::EndCustomFootnote()
+{
+    m_bHasFootnoteStyle = false;
+    m_bCheckFootnoteStyle = false;
+}
 
 void DomainMapper_Impl::PushAnnotation()
 {
@@ -2330,6 +2359,7 @@ void DomainMapper_Impl::PopFootOrEndnote()
     m_aRedlines.pop();
     m_bSeenFootOrEndnoteSeparator = false;
     m_bInFootOrEndnote = false;
+    m_pFootnoteContext = nullptr;
     m_bFirstParagraphInCell = m_bSaveFirstParagraphInCell;
 }
 
diff --git a/writerfilter/source/dmapper/DomainMapper_Impl.hxx b/writerfilter/source/dmapper/DomainMapper_Impl.hxx
index 813d45bab770..92cef81c495c 100644
--- a/writerfilter/source/dmapper/DomainMapper_Impl.hxx
+++ b/writerfilter/source/dmapper/DomainMapper_Impl.hxx
@@ -484,12 +484,14 @@ private:
     }                               m_eInHeaderFooterImport;
     bool                            m_bDiscardHeaderFooter;
     bool                            m_bInFootOrEndnote;
+    PropertyMapPtr m_pFootnoteContext;
+    bool m_bHasFootnoteStyle;
+    bool m_bCheckFootnoteStyle;
     /// Did we get a <w:separator/> for this footnote already?
     bool                            m_bSeenFootOrEndnoteSeparator;
 
     bool                            m_bLineNumberingSet;
     bool                            m_bIsInFootnoteProperties;
-    bool                            m_bIsCustomFtnMark;
 
     RubyInfo                        m_aRubyInfo;
     //registered frame properties
@@ -743,6 +745,16 @@ public:
     void PushFootOrEndnote( bool bIsFootnote );
     void PopFootOrEndnote();
     bool IsInFootOrEndnote() const { return m_bInFootOrEndnote; }
+
+    void StartCustomFootnote(const PropertyMapPtr pContext);
+    void EndCustomFootnote();
+    bool IsInCustomFootnote() const { return m_bHasFootnoteStyle; }
+    bool CheckFootnoteStyle() const { return m_bCheckFootnoteStyle; }
+    void SetHasFootnoteStyle(bool bVal) { m_bHasFootnoteStyle = bVal; }
+    void SetCheckFootnoteStyle(bool bVal) { m_bCheckFootnoteStyle = bVal; }
+
+    const PropertyMapPtr& GetFootnoteContext() const { return m_pFootnoteContext; }
+    OUString StripFootnoteLabelCharacters(const OUString sAppendString);
     /// Got a <w:separator/>.
     void SeenFootOrEndnoteSeparator();
 
@@ -869,9 +881,6 @@ public:
     void SetInFootnoteProperties(bool bSet) { m_bIsInFootnoteProperties = bSet;}
     bool IsInFootnoteProperties() const { return m_bIsInFootnoteProperties;}
 
-    void SetCustomFtnMark(bool bSet) { m_bIsCustomFtnMark = bSet; }
-    bool IsCustomFtnMark() const { return m_bIsCustomFtnMark;  }
-
     bool IsInComments() const { return m_bIsInComments; };
 
     void CheckUnregisteredFrameConversion( );
diff --git a/writerfilter/source/dmapper/PropertyMap.hxx b/writerfilter/source/dmapper/PropertyMap.hxx
index 08a451ed011d..64e9969fa038 100644
--- a/writerfilter/source/dmapper/PropertyMap.hxx
+++ b/writerfilter/source/dmapper/PropertyMap.hxx
@@ -126,6 +126,7 @@ private:
     // marks context as footnote context - ::text( ) events contain either the footnote character or can be ignored
     // depending on sprmCSymbol
     css::uno::Reference< css::text::XFootnote > m_xFootnote;
+    OUString m_sFootnoteCharStyleName;
     std::map< PropertyIds, PropValue >          m_vMap;
     std::vector< RedlineParamsPtr >             m_aRedlines;
 
@@ -154,8 +155,13 @@ public:
     bool isSet( PropertyIds eId ) const;
 
     const css::uno::Reference< css::text::XFootnote >& GetFootnote() const { return m_xFootnote; }
+    const OUString& GetFootnoteStyle() const { return m_sFootnoteCharStyleName; }
 
-    void SetFootnote( const css::uno::Reference< css::text::XFootnote >& xF ) { m_xFootnote = xF; }
+    void SetFootnote(const css::uno::Reference< css::text::XFootnote >& xFootnote, const OUString& sStyleName)
+    {
+        m_xFootnote = xFootnote;
+        m_sFootnoteCharStyleName = sStyleName;
+    }
 
     virtual void insertTableProperties( const PropertyMap*, const bool bOverwrite = true );